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OPNAV  NOTICE  3905 

From:  Chief  of  Naval  Operations 

Sub j :  Fifth  Annual  Report  on  the  Utilization  of  People-Related 
RDT&E 

Ref:  (a)  UNDSECNAV  Memo  of  26  Sept  1978,  Subj:  Navy  Manpower, 

_  Personnel  and  Training  Research,  Development  and 

*•  Studies  (NOTAL) 

(b)  General  Accounting  Office  Report  FPCD  77-43  of  22  Apr 
1977  (NOTAL) 

Enel:  (1)  Fifth  Annual  Report  on  the  Utilization  of  People- 

Related  Navy  RDT&E 

1.  Purpose .  To  promulgate  the  Fifth  Annual  Report  on  the  Utili¬ 
zation  of  People-Related  Navy  RDT&E  (enclosure  (1))  and  forward 
it  for  information  and  appropriate  action.  This  report  contains 
examples  of  the  use  of  this  R&D.  It  is  a  principal  means  whereby 
the  Navy  complies  with  the  recommendations  of  references  (a)  and 
(b)  that  utilization  be  encouraged,  that  communication  between 
the  user  and  researcher  be  improved,  and  that  utilization ~of  this 
R&D  be  tracked. 

2.  Action 

a.  Addressees  are  requested  to  review  enclosure  (1)  to  iden¬ 
tify  any  of  the  following : 

U)  RDT&E  that  they  have  used  but  has  not  been  reported 
in  enclosure  (1)  or  any  of  its  predecessors. 

(2)  Completed  RDT&E  that  might  be  useful  in  activities 
under  their  cognizance. 

(3)  RDT&E  near  completion  that  they  should  monitor  to  use 
when  completed. 

b.  Any  previously  unreported  instances  of  utilization  should 
be  reported  to: 

Commanding  Officer 

Navy  Personnel  Research  and  Development  Center 
Code  303  Wfan  I 

San  Diego,  CA  92125  - T  I 
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e.  Addressees  desiring  naaiatanco  in  using  completed  RDT&E 
°r  fP  monitoring  potentially  useful  Kto  should  direct  inquiries 
to  the  appropriate  R&d  organisations  as  listed  in  enclosure  (D* 

d.  Request  appropriate  action  to  be  completed  by  31  May  1983. 
Negative  reports  are  not  required. 

8*  Report .  Symbol  OPNAV  3905-1  has  been  assigned  to  the  require¬ 
ment  contained  in  paragraph  2.  This  information  will  be  requested 
yearly  by  notice. 
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FOREWORD 


This  is  the  fifth  annual  report  on  people-related 
research,  development,  test,  and  evaluation  (RDT&E)  in 
the  Navy.  Since  the  first  report  in  the  series  was  pub¬ 
lished  in  1978,  covering  fiscal  years  1975-1977,  a  con¬ 
siderable  number  of  accomplishments  in  the  people- 
related  areas  of  manpower,  personnel,  training,  and 
human  factors  engineering  have  been  described.  These 
have  included  large-scale,  multi-year  programs  as  well 
as  smaller  project  efforts  in  key  areas. 

The  p{jpiary  emphasis  in  these  reports  has  been  on 
the  utilization  of  R&O  products  by  Navy  operational 
communities,  and  on  the  benefits  in  both  cost  and 
operational  effectiveness  resulting  therefrom.  An  addi¬ 
tional  focus  has  been  on  the  significant  advances  in  the 
state  of  the  art  that  many  of  these  R&D  efforts  have 
contributed,  ft  is  possible  for  one  who  reads  each  of  the 
utilization  reports  closely  to  note  instances  in  which 
earlier  efforts  have  laid  the  groundwork  for  later,  related 
developments.  This  building  process  is  crucial  to  the 


people-related  RDT&E  program,  as  it  is  in  any  scientific 
or  technical  endeavor. 

The  projects  described  here  represent  only  a  fraction 
of  the  overall  program  of  people-related  RDT&E  being 
conducted  at  centers  and  laboratories  throughout  the 
Navy.  The  efforts  reported  are  not  unique;  they  are 
instead  representative  of  people-related  R&D  products 
utilized  in  FY81  or  being  prepared  for  near-term 
implementation. 

This  report  was  compiled  by  the  Navy  Personnel 
Research  and  Development  Center.  San  Diego.  The 
manuscript  was  prepared  by  Resource  Consultants. 
Inc.,  of  McLean,  VA.  under  Contract  N66001-81-C- 
0432.  Appreciation  is  expressed  to  participating  per7" 
sonnel  at  each  contributing  command,  and  also  to 
those  individuals  who  contributed  documentation,  clari¬ 
fication,  and  sound  advice  while  the  report  was  in  prep¬ 
aration.  Inquiries  and  comments  from  sponsoring  and 
user  commands  are  encouraged. 
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INTRODUCTION 


The  Setting 

In  his  statement  before  the  Subcommittee  on  Man¬ 
power  and  Personnel  of  the  Senate  Armed  Services 
Committee  on  March  4,  1982,  the  Deputy  Chief  of 
Naval  Operations  (Manpower,  Personnel  and  Training) 
(OP-OI )  commented: 

"...  Navy's  manpower,  personnel  and  training 
challenge  of  the  80's  is  to  man  a  growing  Navy 
with  sufficient  manpower  of  the  proper  skill  mix. 
We^pce  this  challenge  during  a  period  of  increas¬ 
ing  national  commitments,  a  high  tempo  of  fleet 
operations,  and  a  declining  manpower  pool  of  18- 
21  year  old  males." 

This  challenge  is  expressed  in  differing  manner  from 
time  to  time,  and  its  exact  character  changes  as  there 
are  adjustments  in  external  factors  such  as  the  econ¬ 
omy.  Nonetheless,  there  is  no  reason  to  expect  relief 
from  its  influence.  In  peacetime,  the  competing  inter¬ 
ests  within  and  dynamic  nature  of  our  society  are  such 
that  resource  issues  are  not  necessarily  resolved  in 
favor  of  national  defense.  On  the  other  hand,  those  who 
are  charged  with  national  defense  cannot  relax  their 
concern  for  the  readiness  of  the  force.  In  essence,  we 
must  make  the  absolute  best  possible  use  of  the 
resources  we  are  provided. 

The  Training  and  Personnel  Systems  Technology 
Program 

Research  and  development  is  the  enterprise  in  which 
we  engage  as  a  means  to  improve  the  use  of  our 
resources  in  meeting  future  needs.  In  the  case  of  the 
men  and  women,  civilian,  military  and  Reserve,  who 
people  the  Navy,  the  research  and  development  pro¬ 
gram  is  titled  within  the  Department  of  Defense  as  the 
"Training  and  Personnel  Systems  Technology  (TPST) 
Program."  The  totality  of  the  TPST  Program  in  all  the 
services  approximates  slightly  more  than  1%  of  the 
Defense  research  and  development  program. 

The  TPST  Program  develops  and  initiates  implemen¬ 
tation  of  new  techniques  in  four  categories  related  to 
the  effective  use  of  our  human  resources.  These  four 
categories,  which  form  the  organizational  structure  of 
this  report,  are: 

•  Manpower  and  Personnel 
O  Education  and  Training 

•  Human  Factors  Engineering 

•  Simulation  and  Training  Devices 

With  changes  in  the  policy  and  economic  environ¬ 
ments.  in  the  structure  of  the  operational  forces,  and  in 


the  demographic  and  sociological  makeup  of  the  nation 
as  a  whole,  particular  areas  of  effort  at  times  assume 
greater  relative  importance.  The  emphases  in  TPST  in 
the  period  of  this  report  have  been  on  efforts  to: 


•  Maintain  performance  levels 

•  Predict  the  effectiveness  of  personnel 

•  Develop  lower-cost  training  that  is  more  effective 
and  more  available 

•  Build  the  technology  base  to  meet  future  needs 

•  Make  greater  use  of  available  microprocessor 
technology 

•  Enhance  the  impact  of  manpower,  personnel  and 
training  in  the  weapon  system  acquisition  cycle 

The  Purpose  of  the  Utilization  Report 

This  report  is  one  means  by  which  the  Navy  commun¬ 
icates  its  success  in  developing  and  implementing  spe¬ 
cie  solutions  to  problems  the  service  faces  in  the 
acquisition,  training,  utilization,  and  management  of 
personnel.  The  report  is  intended  to  increase  the  utiliza- 
t  on  of  newly  developed  training  and  personnel  systems 
technology,  and  also  to  foster  greater  Navy-wide  and 
ir.terservice  coordination  of  efforts  in  people-related 
FOT&E.  In  each  project  summary,  emphasis  is  placed 
on  the  operational  need  that  generated  the  effort,  the 
manner  in  which  the  research  products  or  results  were 
(c  will  be)  utilized,  and  the  actual  or  potential  "payoff". 
Each  summary  also  includes  a  section  which  identifies 
tea  performing  activity,  sources  of  funding  for  the  proj¬ 
ect,  and  sources  of  additional  information.  A  diagram  is 
P'ovided  illustrating  the  evolution  of  the  effort  through 
research,  development  and  implementation.  The  suc¬ 
cessive  levels  are: 

•  6. 1:  Research  —  Scientific  study  and  experimen¬ 
tation  directed  toward  increasing  knowledge  in 
fields  related  to  longterm  national  security  needs. 

•  6.2:  Exp/oratory  Development  —  Efforts  directed 
toward  the  solution  of  specific  military  problems, 
short  of  major  development. 

•  6.3:  Advanced  Development  —  Development  of 
systems  or  system  components  for  experimental 
or  operational  test.  A  primary  objective  is  proof  of 
design  concepts. 

•  6.4  Engineering  Development  —  Design,  fabri¬ 
cation,  and  testing  of  full-scale  systems  for  opera¬ 
tional  application. 
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Emphases  in  this  Report 

The  Utilization  Report  for  FY81  focuses  on  work 
which  has  been  completed  recently  and  which  empha¬ 
sized  improvements  such  as. 

0  Making  the  compensation  system  more  effective 

0  Providing  greater  accuracy  in  predictions  of  per¬ 
sonnel  requirements 

<9  Determining  more  accurately  why  people  stay  in 
the  service  and  why  they  leave  the  service 

O  Utilizing  recruiting  resources  more  effectively 

9  Getting  more  information  from  the  tests  that  are 
administered  to  people  for  classification  and 

'  training  purposes 

C  leaking  better  use  of  computer  systems  to  im¬ 
prove  training  effectiveness  and  efficiency 

O  Improving  procedures  for  training  in  the  basic 
skills  of  mathematics 


•  Adjusting  training  pipelines  so  that  there  is  less 
training  cost  at  the  beginning  of  the  first 
enlistment 

0  Reducing  the  costs  involved  in  the  preparation  of 
software  documentation 

0  Designing  equipment  on  the  basis  of  "how" 
maintainers  accomplish  maintenance  tasks 

•  Using  energy  resources  more  efficiently 

•  Achieving  a  better  integration  of  men  end  equip¬ 
ment  in  system  performance 

•  Reducing  the  cost  and  increasing  the  safety  of 
fire  fighting  training 

•  Improving  the  skills  and  reducing  the  cost  of  train¬ 
ing  for  gunners  firing  an  anti-armor  weapon 

•  Expanding  the  use  of  voice  technology  ;o  reduce 
the  costs  of  training  and  improve  the  perform¬ 
ance  of  systems  operators 
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MANPOWER  AND  PERSONNEL 


IntKe  Department  of  Defense,  Manpower  and  Personnel  RDT&E  involves  "Development  of  techniques/methods 
for  utilizing  available  personnel  resources  through  improved  selection,  job  assignment,  organizational  analysis  and 
management  techniques  to  meet  combat-available  and  projected  force  needs." 

The  Navy  must  continually  improve  its  manpower  and  personnel  processes.  These  processes  include:  estimating 
life-cycle  manpower  requirements;  developing  more  effective  procedures  for  acquiring  and  classifying  personnel; 
increasing  productivity;  and  maintaining  management  capability  to  respond  and  adapt  effectively  to  a  changing  force 
structure.  A  major  focus  at  the  present  time  is  on  finding  ways  to  increase  productivity  and  optimize  the  use  of 
personnel  resources  while  reducing  or  restraining  costs. 

Projects  reported  this  year  are: 

•  Methods  for  Adjusting  Military  Pay 

•  Determining  Shore  Activity  Workload  from  Fleet  Configuration 

•  Survival  Tracking  System 

•  Improving  the  Allocation  of  Recruiting  Resources 

•  Reasons  for  Separation 

•  Improving  the  Efficiency  of  Psychological  Testing 
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METHODS  FOR  ADJUSTING  MILITARY  PAY 


Need 

Annual  increases  in  military  pay  are  a  relatively 
recent  development.  They  began  in  1963  but,  until 
1967,  were  made  by  the  Congress  when  military  pay 
was  perceived  as  having  fallen  behind  federal  civilian 
pay.  In  1967,  Public  Law  90-207  "indexed"  changes  in 
military  wages  to  changes  in  the  General  Schedule 
(GS)  salaries  for  civilian  employees  of  the  federal 
government.  However,  since  1962,  GS  salaries  have 
been  based  on  the  principle  of  "comparability".  Concep¬ 
tually,  this  means  that  GS  salaries  are  set  approxi¬ 
mately  equal  to  civilian  salaries  for  similar  jobs.  Further, 
GS  salaries  are  supposed  to  be  adjusted  over  time  at  a 
rate  egpal  to  changes  in  those  same  civilian  salaries. 
These  changes  are  guided  by  the  annual  survey  of  Pro¬ 
fessional,  Administrative,  Technical,  and  Clerical  (PATC) 
workers  conducted  by  the  Bureau  of  Labor  Statistics 
(BLS).  This  "automatic"  adjustment  mechanism  has 
resulted  in  a  systematic  procedure  for  adjusting  regular 
military  compensation  (RMC). 

In  1980,  Congress  temporarily  severed  the  link 
between  military  and  federal  civilian  pay  by  granting  an 
1 1 .7  percent  military  pay  increase,  as  compared  to  a 
9.1  percent  civilian  raise.  At  the  same  time.  Congress 
questioned  whether  the  PATC  index  and  the  resulting 
increase  in  GS  salaries  was  an  appropriate  guide  for 
adjusting  military  pay  and,  if  not,  what  was  appropriate. 
Prior  to  responding,  the  Assistant  Secretary  of  Defense 
(Manpower,  Reserve  Affairs  and  Logistics)  requested 
recommendations  from  the  services.  The  Deputy  Chief 
of  Naval  Operations  (Manpower,  Personnel  and  Train¬ 
ing)  (OP-01)  was  responsible  for  development  of  the 
Navy  recommendations. 


Approach  and  Results 

The  use  of  the  PATC  index  to  adjust  military  pay 
implicitly  assumes  that  the  occupations  it  surveys  reflect 
job  alternatives  available  to  military  members.  How¬ 
ever,  PATC  concentrates  on  white-collar  occupations. 
Since  most  of  the  occupations  of  the  enlisted  force  are 
of  a  blue-collar  character,  PATC  is  estimated  to  repre¬ 
sent  only  about  10  percent  of  the  enlisted  force.  There¬ 
fore,  the  need  was  to  identify  an  index  which  fits  better 
than  PATC.  Alternative  indexes  were  identified  and 
compared  to  PATC.  Criteria  for  evaluation  included  the 
relative  similarity  of  jobs  in  the  military  to  those  con¬ 
tained  in  the  wages  indexes;  the  timeliness  and  quality 
of  the  data  contained  in  the  indexes;  and  the  ability  of 
the  indexes  to  maintain  real  purchasing  power  (i.e„  to 
keep  up  with  inflation). 


Five  indexes  stood  above  the  others  relative  to  the 
criteria.  They  were,  the  PATC  index;  a  combination  of 
the  PATC  and  the  "blue  collar"  composite  (BC)  indexes; 
a  combination  of  the  PATC  and  area  wage  surveys 
(AWS)  indexes;  the  private,  non-agricultural  production 


and  nonsupervisory  workers  earnings  (PVT/  NONAG) 
index;  and  an  age-earnings  profile  index. 

As  compared  to  the  10  percent  enlisted  coverage  of 
PATC,  the  PATC/AWS  index  accounts  for  approximately 
60  percent  of  enlisted  force  jobs  and  22  percent  of  the 
officer  occupations.  The  PATC  and  PVT/NONAG  indexes 
are  considerably  more  timely  than  the  other  indexes, 
reflecting  wage  increases  of  6  months  prior  or  less.  In 
contrast.  PATC/AWS  data  would  be  15  months  old 
when  used  to  adjust  military  pay.  Moreover,  since  1 972 
all  wages  represented  by  these  indexes  have  suffered 
some  loss  in  real  purchasing  power  as  measured  by  the 
Consumer  Price  Index  (CPI).  The  chart  which  follows 
illustrates  PATC/  AWS  and  PATC  as  compared  to  the 
CPI,  using  PATC  as  the  baseline  because  it  is  the  legis¬ 
lated  index.  RMC  is  also  shown,  illustrating  that  the 
PATC  index,  filtered  through  GS  salaries,  has  not  main¬ 
tained  a  consistent  relationship  to  CPI. 


MAINTAINING  PURCHASING  POWER 


Utilization 

The  results  of  this  effort  were  used  in  an  April  1 981 
OSD  Pay  Study  convened  to  consider  pay  adjustment 
mechanisms.  They  were  also  used  in  a  Pay  Study 
beginning  in  February  1 982.  That  Pay  Study  is  expected 
to  recommend  to  Congress  an  alternative  to  the  PATC 
index. 


Impact 

One  vital  component  of  military  retention  incentives 
is  annual  pay  raises.  However,  to  be  effective,  these  pay 
raises  must  be  competitive  with  wage  growth  in  civilian 
occupations  with  which  the  military  competes  for  tal¬ 
ent.  This  research  and  development  has  identified  a  pay 
adjustment  mechanism  more  representative  of  military 
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jobs  —  an  index  that  accurately  reflects  relevant  labor 
market  forces. 


Research  and  Development  Notes 

This  work  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC)  under  the 
sponsorship  of  the  Deputy  Chief  of  Naval  Operations 
(Manpower,  Personnel  and  Training)  (0P-01).  The  proj¬ 
ect  identification  was  “Methods  for  Adjusting  Military 
Pay",  Z1182-PN.03,  in  Program  Element  63707N, 
Manpower  Control  System  Development.  The  respon¬ 
sible  researcher  was  M.  W.  Rowe,  NPRDC  (Code  11), 
(714)  225-7388.  Recent  Technical  Reports  include 
NRPDC  TR  81  -1,  Review  and  Analysis  of  the  Legislative 
History/Intent,  Cost,  and  Value  of  Special  Pay  While  on 
Duty  'at  Certain  Places.  November  1980:  TR  82-12, 
Methods  for  Adjusting  Military  Pay,  October  1 981;  TR 
82- 1  l.Cdftain  Places  Pay:  Current  Inconsistencies  and 
Suggested  Alternatives. November  1981;  and  TR  82- 
(),The  Reallocation  of  Military  Pay  Increases.in  press. 

Program  dynamics  are: 


r 
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DETERMINING  SHORE  ACTIVITY  WORKLOAD  FROM  FLEET  CONFIGURATION 


Need 

The  United  States  Navy  has  a  complex  system  of 
more  than  1000  shore  activities  supporting  its  ships 
and  aircraft.  In  addition,  the  shore  activities  must  sup¬ 
port  each  other.  Some  of  the  major  activities  include 
hospitals,  shipyards,  airfields,  aircraft  repair  facilities, 
research  laboratories,  supply  depots,  activities  which 
maintain  and  repair  Navy  buildings  and  roads,  and 
many  training  bases. 


One  of  the  Navy's  most  intractable  problems  has 
been  to  forecast  the  effects  that  changes  in  fleet  size  or 
configuration  have  on  the  workload  of  these  shore 
activities  and  their  requirements  for  personnel  to 
accomplish  the  workload.  This  effect  must  be  forecast 
so  that  the  people  in  the  right  skills  and  number  are 
onboard  the  activity  when  the  workload  changes.  The 
support  personnel  requirements  are  not  only  affected 
by  the  size  and  mix  of  the  fleet,  but  also  by  its  operating 
tempo  and  by  indirect  fleet  demands  channelled  through 
other  shore  activities.  Conversely,  the  Navy  must  also 
be  able  to  detetmine  the  mix  of  fleet  elements  that  can 
be  supported  by  the  shore  establishment  as  it  exists  at 
the  time  in  question.  For  example,  in  a  past  program 
analysis  one  issue  centered  on  the  increase  of  6%  per 
year  (after  inflation)  in  operations  and  maintenance 
funds  during  FYs  1 973  through  1 978,  while  the  number 
of  ships  was  decreasing.  An  inability  to  respond  quan¬ 
titatively  to  questions  of  this  nature  leads  to  pro  rata 
personnel  reductions  at  all  shore  activities,  which  usu¬ 
ally  results  in  uneven  levels  of  fleet  support 


Approach  and  Results 

This  research  initially  developed  a  computerized 
interactive  input-output  (I/O)  model  (CIOM,  of  the  fleet- 
support  demand  network  of  the  11th  Naval  District 
(11ND).  CIOM  was  discussed  in  the  Fiscal  Year  (FY) 
1980  Annual  Report  on  The  Utilization  of  People- 
Related  Navy  RDT&E.  CIOM  can  be  used  to  forecast 
workload  at  nine  major  1 1 ND  shore  activities,  given  a 
specified  fleet  configuration,  and  to  estimate  the  per¬ 
centage  changes  from  base-year  workload  for  these 
activities. 

Next,  an  I/O  model  was  developed  for  supply  support 
to  the  Pacific  Fleet.  Data  were  collected  and  analyzed 
for  Naval  Supply  Centers  (NSCs)  San  Diego,  Oakland, 
Puget  Sound,  and  Pearl  Harbor;  Naval  Supply  Depots 
(NSDs)  Yokosuka,  Subic  Bay,  and  Guam;  Combat  Store 
Ships  (AFSs):  and  Submarine  Tenders  (ASs).  Major 
emphasis  was  given  to  analyzing  fleet  demands  by  ship 
class,  employment  status,  and  homeport.  Further,  Navy 
shore  activities  were  organized  in  an  I/O  matrix  into 
workload  sectors,  so  that  the  extent  to  which  each 
activity  depends  on  every  other  activity  to  produce  sup¬ 
port  could  be  quantified  using  historical  data. 

The  Pacific  Fleet  I/O  Model  was  used  to  measure  the 
effects  of  increased  deployment  in  the  Arabian  Sea  and 
Indian  Ocean  area.  The  model  determined  direct  and 
indirect  effect  on  supply  workload  at  each  Pacific  Fleet 
supply  activity  in  terms  of  the  number  of  requisitions. 
For  example,  it  determined  the  workload  increase  at 
NSC  Oakland  due  to  its  role  in  support  of  the  deployed 
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ships.  In  addition,  the  model  determined  the  indirect 
workload  increase  at  NSC  Oakland  due  to  increased 
demands  on  the  supply  depots  and  repair  facilities 
which  NSC  Oakland  supports  in  the  Western  Pacific. 

Early  in  FY  1981,  the  Pacific  Fleet  I/O  Model  was 
verified  using  actual  FY  1978  workload  data.  In  addi¬ 
tion,  the  impact  that  deployment  patterns  have  on  the 
number  of  supply  requisitions  was  tested  using  actual 
FY  1980  ship  employment  data.  As  the  figure  below 
illustrates  for  NSD  Subic  Bay,  the  ability  of  the  I/O 
model  to  forecast  workload,  given  fleet  configuration 
and  operating  tempo,  was  validated. 


- ACTUAL 

—  —  PROJECTED 


SO  .  81 

FISCAL  YEAR 

NSO  SUBIC  BAY 

Actual/projected  percent  workload 
changes - FY78-85 


“Logistics  support  requirements  for  the  Pacific 
Fleet,  obviously  a  current  concern,  will  be  a  major 
topic  of  review  in  the  near  future.  As  the  OPNAV 
staff  prepares  to  develop  its  Program  Objectives 
Memorandum  for  FY  83  -  FY  87  (POM-83),  the 
insights  to  be  gained  from  ...  (The  PACFLT  I/O 
Model)  are  likely  to  yield  real-world  resource  allo¬ 
cation  decisions. ...  we  are  anticipating  additional 
inputs  during  the  coming  months.” 

Expansion  of  the  Pacific  Fleet  Input-Output  Model  to 
other  functions  and  regions  can  be  undertaken  when 
funded.  Extensions  to  other  logistic  functions,  such  as 
maintenance,  would  allow  more  accurate  determina¬ 
tion  of  the  size  of  shore  establishment  (in  terms  of 
workload  and  manpower)  needed  to  support  a  specified 
fleet  configuration  (number  of  ships,  mix,  operating 
tempo),  and  in  determining  what  alternative  fleet  con¬ 
figurations  a  given  shore  establishment  could  support. 


Research  and  Development  Notes 

This  work  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  under  the  sponsor¬ 
ship  of  the  Deputy  Chief  of  Naval  Operations  (Man¬ 
power,  Personnel,  and  Training)  (OP-01).  The  project 
identification  was  “Determining  Shore  Activity  Work¬ 
load  from  Fleet  Configuration",  Z1186-PN.02.  in  Pro¬ 
gram  Element  63707N,  Manpower  Control  System 
Development.  The  responsible  researcher  was  M.  R- 
Shoecraft,  NPRDC(Code  1 1).  (71 4)  225-2971.  The  proj¬ 
ect  has  been  documented  in  NPRDC  Technical  Note 
80-16,  Modeling  Logistics  Support  Requirements  for 
the  Pacific  Fleet,  and  Technical  Report  82-1,  Supply 
Workload  Implications  of  Increased  Deployment  to  the 
Indian  Ocean. 


(r 


Utilization 

The  Pacific  Fleet  I/O  Model  was  used  to  make  work¬ 
load  projections  for  FY  1 981  and  FY  1 985  for  the  seven 
NSCs  and  NSDs.  The  forecasts  demonstrated  the 
model's  usefulness  for  resource  allocation  decisions 
and  its  versatility  in  determining  the  effects  of  changing 
fleet  configuration  and  operating  tempo  on  supply 
resources  in  the  Pacific  Fleet.  The  projections  were 
used  by  supply  resource  sponsors  in  development  of 
Program  Objectives  Memorandum  (POM)  1 983. 

Fleet  requisition  demand  rates  of  ships  by  type  (e.g., 
destroyer,  frigate)  and  status  (in  port,  deployed,  or  in 
overhaul)  which  were  developed  for  the  I/O  model 
were  also  used  by  the  Naval  Supply  Systems  Command 
and  Naval  Supply  Centers  as  planning  factors  in  making 
resource  allocation  decisions. 


Payoff 

The  Assistant  Deputy  Chief  of  Naval  Operations 
(Logistics)  (OP-40)  expressed  appreciation  for  the  work 
to  the  Commanding  Officer,  NPRDC. 


Program  dynamics  are: 


APPLICATION 

POTENTIAL 

Effect  of  increased 

Development  ol  a 

deployment  to 

Navy-wide  supply 

Indian  Ocean  on 

vid  maintenance 

navsup/pacflt 

|  workload 

Resources. 

forecasting  model. 

Impact  of  600  ship 

Navy  on  supply  and 

maintenance  workload. 

6.3  ADVANCED  DEVELOPMENT  ^ 

Development  of  •  computerized  input-output  model  of  the 
fleet  .support  demend  network  ol  the  Eleventh  Navel  District. 

Development,  test  and  evaluation  of  a  model  to  forecast  work¬ 
load  (requisitions)  at  all  Pacific  Fleet  Supply  Activities. 
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SURVIVAL  TRACKING  SYSTEM 


Need 

Many  policies  and  programs  in  areas  such  as  com¬ 
pensation,  recruitment,  and  training  are  implemented 
by  the  Navy  to  influence  the  retention,  training,  perform¬ 
ance,  or  career  progress  of  enlisted  personnel.  All  too 
often,  however,  data  which  track  the  behavior  of  indi¬ 
viduals  affected  by  these  policies  are  missing,  incom¬ 
plete  or  difficult  to  obtain.  This  deficiency  renders  a 
comprehensive  evaluation  of  policies  and  programs 
expensive  and  in  some  cases  impossible  to  achieve. 

Representatives  from  the  Naval  Military  Personnel 
Command  (NMPC)  requested  that  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC)  assist  in 
the  development  of  a  comprehensive  automated  sys¬ 
tem  which  tracks  the  progress  of  Navy  personnel 
throughout  their  enlisted  careers  and  permits  quick 
retrieval  of  key  information  needed  to  evaluate  new  or 
existing  policies  and  programs  in  an  economical  fashion. 


Approach  and  Results 

Informational  requirements  relevant  to  personnel 
policy  decision  making  were  identified.  These  require¬ 
ments  included: 


was  integrated  into  the  system  to  provide  rapid,  eco¬ 
nomical  access  to  key  policy-relevant  data. 

The  result  is  an  operational  system  which  may  be 
expanded  to  included  relevant  data  from  additional 
sources  as  the  need  arises. 


Utilization 

The  initial  operational  system  is  installed  at  the 
Argonne  National  Laboratory  computer  facility  at 
Argonne,  Illinois.  The  system  is  accessible  by  NPRDC 
and  NMPC  personnel.  It  has  been  used  to  obtain  infor¬ 
mation  related  to  issues  such  as  the  impact  of  the 
programmed-school-input  program  on  attrition,  the 
effect  of  Project  Upgrade  on  Navy  strength  planning, 
and  the  potential  implications  of  a  new  female  general 
detail  program. 

Guidance  on  the  content  and  utilization  of  the  system 
has  been  provided  by  NPRDC  to  personnel  in  NMPC  and 
the  Office  of  the  Deputy  Chief  of  Naval  Operations 
(Manpower,  Personnel  and  Training)  (OP-01).  Periodic 
assistance  concerning  system  documentation  and  up¬ 
grading  continues. 


•  Longitudinal  tracking  of  individuals  so  that  the 
impact  of  policies  can  be  evaluated  over  the 
entire  career  continuum. 

•  Data  pertaining  to  significant  career  milestones 
and  key  decisions  in  order  to  monitor  the  effect  of 
policies  upon  various  events  (e.g.,  advancement, 
attrition,  reenlistment,  assignment). 


When  required,  follow-on  research  can  develop  sys¬ 
tem  enhancements  which  will  provide  policy  makers 
with  ready  access  to  an  expanded  set  of  information 
pertaining  to  retention  and  career  development. 


Research  and  Development  Notes 


•  Information  identifying  individuals  as  members  of 
groups  upon  which  personnel  policies  have  been 
or  may  be  focused.  Such  information  includes 
age  and  sex.  time  of  entry,  test  scores,  term  and 
type  of  enlistment  contract,  and  others. 

•  Accuracy,  portability,  maintainability,  and  econ¬ 
omy  of  the  system  so  that  policy  evaluation  is 
based  upon  current  facts  and  figures  and  may  be 
performed  at  minimal  cost. 


Next,  existing  sources  of  Navy  information  were 
evaluated.  Data  sources  (e.g..  Enlisted  Master  File, 
audit  file)  were  chosen  as  input  for  the  system.  Quar¬ 
terly  extracts  from  these  sources  were  derived  which 
contained  the  data  elements  essential  for  policy  evalua¬ 
tion.  These  items  were  selected  on  the  basis  of  consul¬ 
tations  with  Navy  organizations  concerned  with  per¬ 
sonnel  policy  formulation,  implementation,  and  evalua¬ 
tion.  A  file  structure  was  then  developed  which  permits 
both  rapid  updating  of  personnel  data  and  easy  adapta¬ 
tion  to  various  data  processing  configurations.  Finally,  a 
subfile  structure  and  supporting  inquiry  mechanism 


Program  dynamics  are  depicted  in  the  following 
diagram: 


APPLICATION 

Operational  Enlisted  Programming  System 
Operational  Cost-Efficient  Retention 
Programs  Policies 


S'  63 


:s: 


S.3  ADVANCED  DEVELOPMENT 


NT 

/  Prototype  Enlisted  Programming  A 

V  Management  Systems  J 

XsDesign  of  Cost- Eff idem  Retention  Policiejx' 


6.2  EXPLORATORY  DEVELOPMENT 


IS 


Navy  Date  Systems  for  Policy/ 
Program  Evaluation 
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This  work  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  with  the  assistance 
of  the  Navy  Military  Personnel  Command.  Funding  was 
provided  in  Exploratory  Development  Program  Element 
62763N,  Personnel  and  Training  Technology.  The 
responsible  researcher  was  Dr.  J.  I.  Borack,  NPRDC 
(Code  15),  (714)  225-7286.  The  system  is  described  in 
NPRDC  TN  81  -11  .The  Enlisted  Survival  Tracking  Fite, 
April  1981 . 


Payoff 

During  FY  81.  numerous  pending  policy  decisions 
within  OP-01  required  data  to  be  available  on  extremely 
short  notice.  NMPC  used  the  survival  tracking  system  to 
provide  required  data  within  hours  at  sharply  reduced 
costs.  For  example,  a  single  request  for  information 
which  previously  had  cost  the  Navy  approximately  eight 
hundred  dollars  to  process  could  be  obtained  for  as  little 
as  ten  dollars.  Furthermore,  the  system  provides  find¬ 
ings  which  are  in  agreement  with  Navy  statistics,  there¬ 
by  obviating  the  need  for  time  and  personnel  for  data 
reconciliation. 


IMPROVING  THE  ALLOCATION  OF  RECRUITING  RESOURCES 


-  Need 

In  1 978,  concerns  were  increasing  about  the  ability 
of  the  All  Volunteer  Force  to  meet  the  defense  man¬ 
power  requirements  of  the  nation.  The  economy  was 
good  and  enlistments  in  the  Armed  Forces  were  down. 

The  quality  of  enlistments  was  also  an  issue  because 
qualifying  test  scores  were  being  questioned  and  the 
definition  of  "high  school  graduation"  was  broadening. 
Further,  the  population  of  18  year  olds  was  reaching  a 
peak  and  forecasts  predicted  a  steady  decline  of  the  age 
for  military  enlistment  (17-21  year  olds)  over  the  next 
decade. 

These  factors  caused  greater  attention  to  the  means 
by  which  personnel  are  recruited  into  the  services. 
However,  the  cost-effectiveness  of  many  existing  re¬ 
cruiting  programs  had  long  been  questioned.  Recruiting 
and  advertising  budgets  for  each  armed  service  were 
identified  as  special  interest  items  by  the  Congress. 
Adve-  :ising  for  enlistments  on  paid  electronic  media 
had  been  authorized  by  Congress  in  1976.  A  prototype 
joint  services  advertising  campaign  had  been  developed 
and  implementation  was  being  debated.  Individual  ser¬ 
vice  requests  for  budget  supplements  to  meet  antici¬ 
pated  enlistment  shortages  proliferated. 

A  need  was  identified  by  the  Principal  Deputy  Assist¬ 
ant  Secretary  of  the  Navy  (Manpower  and  Reserve 
Affairs)  to  examine  marketing  effectiveness  and  to 
quantify  the  relationships  between  enlistment  achieve¬ 
ment  and  marketing  expenditures.  The  vast  majority  of 
these  expenditures  are  concentrated  on  the  mainte¬ 
nance  and  support  of  the  field  recruiting  force  and  on 
various  types  of  paid  advertising.  The  two  objectives 
were: 

O  To  estimate  a  marketing  effort  response  curve  (as 
measured  by  the  number  of  Navy  enlistment  con¬ 
tracts  at  various  quality  levels)  to  overall  changes 
in  the  Navy  advertising  budget,  the  size  of  the 
Navy  recruiting  force,  and  a  combined  change  in 
both  policy  variables. 

®  To  conduct  supplemental  tracking,  data  collection 
and  analyses  so  as  to  reasonably  maximize  the 
information  about  other  recruiting  policies  and 
practices  that  effect  recruiting  success 


Approach  and  Results 

A  one-year  field  experiment  was  designed  as  follows: 

•  Advertising  and  recruiters  were  to  be  varied  in  a 
systematic  and  controlled  way,  independent  of 
sales  in  prior  periods. 

•  A  broad  range  of  each  treatment  variable  was  to 
be  tested  and  treatments  were  to  be  replicated  in 
multiple  markets. 

•  A  substantial  body  of  supplemental  data  was  to 
be  collected  for  each  market  involved  in  the 
experiment,  including  both  sales  and  marketing 
data  for  the  other  (competing)  branches  of  the 
Armed  Forces.  In  addition,  a  survey  research  pro¬ 
gram  was  to  be  conducted  in  selected  markets. 

The  Area  of  Dominant  Influence  (ADI)  was  selected 
ss  the  unit  of  analysis  for  the  experiment.  Individual 
counties  are  assigned  to  ADIs  by  the  electronic  media¬ 
rating  services  based  on  media-use  patterns  of  sampled 
households.  Of  the  over  200  ADIs  in  the  United  States, 
a  subset  was  selected  for  experimental  treatment.  The 
treatment  objective  was  to  expose  individuals  in  the 
treatment  markets  to  the  level  and  type  of  advertising 
and  recruiters  to  which  they  would  be  exposed  under 
the  alternative  budgel  levels  being  tested.  Experimental 
treatments  were  randomly  imposed  in  26  markets.  A 
total  of  43  markets  were  involved  as  either  control  or 
treatment  markets.  In  addition,  12  markets  were  used 
as  the  sample  for  a  survey  to: 

•  Enable  the  identification  of  any  obstruction 
between  the  two  marketing  control  variables, 
recruiters  and  advertising. 

•  Provide  measures  of  the  impact  of  recruiter  and 
advertising  treatment  on  a  range  of  intermediate 
measures.  These  measures  included  awareness 
of  advertising,  recruiter  contact,  attitudes  towards 
8  variety  of  “life  goals",  perceptions  of  the  Navy, 
and  intentions  to  enlist. 

•  Enrich  the  Navy's  understanding  of  the  recruiting 
environment  and  process. 
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To  achieve  these  aims,  respondents  were  sampled  at 
a  variety  of  points  in  the  recruiting  cycle.  Separate, 
independent  samples  were  drawn  at  each  point  in  the 
process.  Questionnaires  were  administered  in  June  of 
1 979  before  any  experimental  intervention  and  again  in 
June  1 980,  nine  months  into  the  experiment.  The  gen¬ 
eral  target  population  questionnaire  was  administered 
by  telephone  using  random  digit  dialing  techniques.  All 
other  questionnaires  were  administered  by  Navy  Re¬ 
cruiting  or  AFEES  test  station  personnel  in  the  form  of 
written  instruments. 

Highly  significant  relationships  between  the  number 
of  recruiters  present  and  the  number  of  enlistment  con¬ 
tracts  were  observed.  Increasing  the  numbers  of  Navy 
recruiters  also  increases  the  Navy's  relative  share  of 
enlistments.  A  four-to-six  month  (earning  period  for 
newly-assigned  Navy  recruiters  was  also  found.  Enlist¬ 
ment  contract  production  for  new  recruiters  during  this 
period  iseaubstantially  less  than  that  of  experienced 
recruiters.  In  addition,  a  loss  of  productivity  was  noted 
for  Navy  recruiters  who  expected  soon  to  leave  recruit¬ 
ing  duty.  In  one  analysis,  enlistment  contract  produc¬ 
tion  fell  off  sharply  beginning  about  one  year  before 
tour  rotation.  Advertising  expenditures  were  observed 
to  have  significant  effects  on  the  number  and  type  of 
enlistment  contracts  obtained.  In  addition,  costs  for 
achievement  of  enlistment  contracts  of  various  types 
through  recruiting  resource  expenditures  were  esti¬ 
mated  for  those  resources  which  had  consistent  and 
significant  effects  on  contracts. 

Recruiters  show  the  strongest  and  most  consistent 
effects  on  enlistment  contracts.  Hence,  investments  in 
recruiter  resources  bear  relatively  less  financial  risk. 
They  are  also  the  most  expensive  resource  input  at  the 
margin.  A  cost  per  enlistment  contract  for  recruiter 
input  was  estimated  at  about  $2,000  compared  with  an 
average  cost  of  marketing  inputs  per  enlistment  con¬ 
tract  of  about  $1,000.  The  marginal  cost  per  contract 
achieved  through  recruiter  input  rises  steadily  with 
"quality''  constraints. 

Advertising  effects  on  enlistment  contracts  are  not  as 
strong  or  consistent  as  those  for  recruiters.  These 
expenditures  are  thus  "riskier".  Nevertheless,  where 
effective,  advertising  is  highly  efficient.  The  lowest 
marginal  cost  estimated  is  for  Navy  local  advertising 
with  respect  to  Navy  high  school  diploma  or  graduate 
contracts.  The  cost  is  estimated  at  $200  per  contract. 
The  marginal  cost  per  Navy  enlistment  contract  for  joint 
services  advertising  expenditures  is  generally  under 
$1,000.  Marginal  costs  for  Navy  national  advertising 
were  not  estimated  because  aggregate  effects  were 
very  small  and  inconsistent  for  this  campaign.  Hence 
marginal  costs  for  this  resource  would  be  exceptionally 
high. 

Utilization  and  Payoff 

These  results  will  enable  the  Navy  Recruiting  Com¬ 


mand  to  allocate  resources  to  advertising  and  to 
recruiters  more  effectively  than  before.  Subsidiary  anal¬ 
yses  will  enable  recruitment  planners  to  target  more 
effectively  those  marketing  campaigns  directed  at  spe¬ 
cific  segments  of  the  market  such  as  Mental  Groups  I 
and  II  or  women.  The  findings  on  learning  and  loss  of 
productivity  are  being  taken  into  account  in  determining 
how  long  recruiters  should  remain  in  a  given  location, 
how  they  can  be  brought  up  to  speed  most  rapidly,  and 
how  their  productivity  can  be  kept  at  a  high  level  during 
the  latter  part  of  their  tour-of-duty.  In  addition,  the 
knowledge  gained  enables  the  Navy  to  justify  its  recruit¬ 
ing  budgets  to  the  Department  of  Defense  and  Con¬ 
gress  much  more  effectively. 


Research  and  Development  Notes 

The  Chief  of  Naval  Research  sponsored  this  project  in 
Program  Element  62763N,  Personnel  and  Training 
Technology.  The  project  was  initiated  in  1978  and  com¬ 
pleted  in  1981.  The  responsible  researcher  was  Dr.  B. 
T.  King,  ONR  (Code  442),  (202)  696-4741 .  The  principal 
investigator  was  Mr.  V.  Carroll,  Wharton  Applied 
Research  Center,  University  of  Pennsylvania.  Philadel¬ 
phia,  PA.  19104,  (215)  243-4470.  Technical  reports  are 
in  preparation. 


Program  dynamics  are: 
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IMPROVING  THE  EFFICIENCY  OF  PSYCHOLOGICAL  TESTING 


Need 

Psychological  tests  play  a  central  role  in  career  selec¬ 
tion  of  all  Navy  and  Marine  Corps  personnel.  In  turn, 
career  opportunities  have  a  substantial  influence  on 
enlistment  decisions,  and  performance  in  training  has  a 
substantial  influence  on  training  costs,  on  training 
motivation,  and  ultimately  on  force  effectiveness.  There¬ 
fore  tests  must  accurately  reflect  an  individual's  capa¬ 
bilities.  Unfortunately,  improved  test  accuracy  generally 
requires  increased  test  length,  which  necessitates  addi¬ 
tional  time. 

However,  both  the  Military  Enlistment  Processing 
Command  (MEPCOM),  which  manages  operations  at 
the  Military  Entrance  Processing  Stations  (MEPS),  and 
Na\%£nd  Marine  Corps  training  commands  are  under 
pressure  to  reduce  testing  time.  For  MEPCOM,  every 
extra  minute  spent  testing  increases  the  probability  that 
candidates  will  need  overnight  accommodations  in 
commercial  hotels.  For  Navy  and  Marine  Corps  training 
commands,  testing  time  represents  a  substantial  frac¬ 
tion  of  training  time,  and  every  man-day  spent  in  train¬ 
ing  is  a  man-day  not  spent  in  line  units  which  are  criti¬ 
cally  short  of  trained  personnel. 

Obviously,  if  the  efficiency  of  psychological  tests 
were  to  be  increased,  more  accurate  testing  could  be 
accomplished  without  increased  time.  One  idea  for 
increasing  the  efficiency  of  tests  is  to  use  a  more 
sophisticated  procedure  (polychotomous  scoring)  for 
scoring  responses.  Scoring  models  currently  used  to 
describe  the  performance  of  individuals  on  multiple- 
choice  or  free-response  tests  are  dichotomous,  an 
examinee  who  produces  a  correct  response  is  scored 
"correct";  an  examinee  who  produces  an  incorrect 
response  is  scored  "incorrect".  No  distinction  is  made 
between  the  various  incorrect  responses.  However,  we 
know  that  examinees  who  do  not  know  the  correct 
answer  are  often  either  misled  by  a  distractor  or  make 
an  informed  choice  using  whatever  partial  information 
they  have.  They  seldom  respond  randomly.  Polycho¬ 


tomous  scoring  would  improve  testing  efficiency  by 
capitalizing  upon  information  in  an  examinee's  choice 
of  incorrect  alternative.  This  has  been  a  popular  idea, 
but  previous  attempts  to  use  polychotomous  scoring 
have  been  disappointing. 

A  relatively  new  approach  to  psychological  measure¬ 
ment  known  as  latent-trait  theory  has  substantially 
improved  the  ability  to  model  correct-answer  choice 
(i.e.,  dichotomous  scoring).  While  there  had  been  some 
work  on  a  polychotomous  latent-trait  model,  the  models 
were  very  rudimentary  and  the  statistical  and  numeri¬ 
cal  procedures  were  impractical.  Therefore,  this  project 
was  undertaken  to  develop  a  comprehensive  latent- 
trait  theory  for  polychotomous  scoring. 


Approach  and  Results 

A  latent-trait  theory  for  the  polychotomous  situation 
needed  two  things:  a  performance  model  which  de¬ 
scribes  test  behavior  as  a  function  of  the  capabilities 
being  measured;  and  procedures  for  examining  the 
quality  of  incorrect  response  alternatives. 

The  work  horses  of  polychotomous  scoring  are 
"response-category  operating  characteristics".  These 
operating  characteristics  are  mathematical  functions 
which  describe,  in  a  probabilistic  way,  the  relationship 
between  an  examinee's  capabilities  and  his  response 
choices.  With  a  five-alternative  multiple-choice  prob¬ 
lem,  one  could  have  as  many  as  five  operating  charac¬ 
teristics,  one  operating  characteristic  for  each 
alternative. 

In  figure  1 ,  the  operating  characteristic  for  response 
option  A,  say.  describes  how  the  conditional  probability 
of  selecting  option  A  varies  as  a  function  of  ability. 
Statistical  and  numerical  difficulties  in  estimating  these 
operating  characteristics  had  stymied  previous 
attempts  to  develop  a  useable  latent-trait  theory  for 
polychotomous-scoring. 


Figure  1 
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If  response-choice  is  to  be  useful  in  improving  the 
efficiency  of  multiple-choice  tests,  it  must  be  informa¬ 
tive.  If  the  operating  characteristic  of  option  A,  say,  is 
very  flat  over  a  broad  range  of  the  individual’s  capabili¬ 
ties,  the  fact  that  an  examinee  chooses  option  A  tells  us 
little  about  his  capabilities;  a  peaked  operating  charac¬ 
teristic  would  be  more  informative.  Ideally,  one  would 
like  to  have  very  peaked  unimodal  operating  character¬ 
istics  for  the  incorrect  responses,  and  a  strictly- 
increasing  operating  characteristic  which  rises  sharply 
to  an  asymptote  for  the  correct  response.  Practically, 
one  needs  procedures  for  gauging  the  performance  of  a 
particular  test  item  against  this  ideal.  This  work  has 
developed  indices  for  that  purpose. 


Technology-Base  Advancement 

Five  years  ago,  useable  latent-trait  theory  consisted 
entirely  of  models  suitable  for  dichotomously-scored, 
free-respense,  unidimensional  test  questions.  Although 
models  for  other  response  formats  and  other  scoring 
schemes  had  been  considered,  formidable  statistical 
and  numerical  estimation  problems  existed.  This  proj¬ 
ect  has  developed  a  latent-trait  theory  for  almost  any 
type  of  discrete-response  unidimensional  test  question. 

Results  of  this  project  have  not  only  been  described 
in  technical  reports  but  also  have  been  implemented  in 
exportable  computer  programs.  Interim  results  have 
been  described  at  annual  ONR  contractor's  meetings 
attended  by  representatives  of  Navy  research  and 
development  activities  and  operational  commands.  In 
addition,  a  three-day  workshop  to  introduce  users  to 
these  results  and  to  the  computer  programs  which 
implement  them  was  held  in  March  1982.  Results  of 
the  work  appear  in  the  proceedings  of  the  1977  and  the 
1 979  Computerized  Adaptive  Testing  Conferences  and 
in  the  proceedings  of  the  Fifth  International  Symposium 
on  Multivariate  Statistics.  Results  will  also  appear  in 
Wainer,  H.  and  Messick.  S.  (eds),  Advances  in  Psycho¬ 
metric  Theory:  A  Festschrift  for  Frederic  M.  Lord.  Hills¬ 
dale,  NJ;  L  Erlbaum  Assoc,  (to  appear). 


Impact 

The  benefits  of  this  project  come  in  terms  of  increased 
accuracy  and  efficiency  of  psychological  measurement. 
These  increases  will  result  from  better  design  of  mea¬ 


surement  tasks  through  the  use  of  the  distractor- 
assessment  technology;  more  efficient  use  of  each 
examinee’s  time  through  the  increased  accuracy  of  the 
information  used  in  adaptive  selection  of  problems 
which  seem  to  be  at  his  level;  and  more  accurate  scor¬ 
ing  of  an  examinee's  responses  through  the  use  of 
models  which  accurately  reflect  response  choice. 


Research  and  Development  Notes 

This  project  was  sponsored  by  the  Chief  of  Naval 
Research.  It  is  titled  “Statistical  Techniques  for  Graded 
Response  Items  in  Adaptive  Psychological  Testing.” 
Funding  was  provided  in  Program  Element  61153N, 
Defense  Research  Sciences.  Subelement  42  Behav¬ 
ioral  and  Social  Sciences.  It  was  initiated  in  1 977  and 
completed  in  1981 .  The  responsible  research  manager 
was  Dr.  C.  E.  Davis,  ONR  Pasadena.  CA.  (213)  795- 
5971.  The  principal  researcher  was  Dr.  F.  Samejima. 
Department  of  Psychology.  University  of  Tennessee. 
Knoxville,  TN,  3791 6.  The  work  is  summarized  in  Same¬ 
jima,  F.  Final  Report:  Efficient  Methods  of  Estimating 
the  Operating  Characteristics  of  Item-response  Catego¬ 
ries  and  Challenge  to  a  New  Model  for  the  Multiple- 
choice  Item.  Knoxville.  Tennessee;  Department  of 
Psychology,  University  of  Tennessee.  November,  1981. 

Program  dynamics  are; 


REASONS  FOR  SEPARATION 


Need 

The  Navy  has  a  continuing  concern  with  the  financial 
and  '  fleet  readiness"  costs  of  personnel  attrition  A 
frequent  strategy  in  the  effort  to  reduce  attrition  rates  is 
to  conduct  special  surveys  of  the  reasons  people  give 
for  leaving  the  service.  However,  these  surveys  are  very 
costly  They  require  a  great  deal  of  time  to  develop, 
administer  in  operational  units,  and  analyze.  Conse¬ 
quently.  there  is  frequently  a  six-month  lag  between 
the  time  the  information  is  needed  and  the  time  the 


final  results  are  available.  An  ongoing  management 
information  system  is  needed  to  eliminate  the  many 
separate  and  "time-bound"  data  collection  efforts.  It 
will  also  provide  a  longitudinal  "norm"  base  against 
which  to  evaluate  trends  and  compare  various  groups 
of  Navy  personnel. 

Approach  and  Results 

Personnel  separating  from  the  Navy  were  asked  to 
write  out  their  reasons  for  leaving  the  service  These 
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free-form  responses  were  analyzed  to  derive  the  most 
frequently  cited  reasons.  Based  on  the  "most  frequent" 
responses,  as  well  as  information  from  previous  sur¬ 
veys,  30  reasons  were  chosen.  Some  of  the  criteria 
used  in  the  selection  of  "reasons  for  leaving"  were 
whether  each  item  was: 

•  Within  the  power  of  the  Navy  to  change 

•  General  without  being  vague 

•  Different  from  other  reasons 

•  A  response  that  personnel  would  normally  give 
when  asked  "Why  are  you  separating?" 

A  questionnaire  has  been  developed  for  administra¬ 
tion  to  ingjyiduals  leaving  the  service.  Each  person  will 
be  asked  to  indicate  how  important  each  of  the  30  rea¬ 
sons  was  in  the  decision  to  separate.  To  facilitate  the 
analysis  of  subgroups,  various  demographic  items  are 
included,  such  as  rating,  pay-grade,  number  of  reen¬ 
listments,  and  marital  status.  A  special  answer  section 


is  included  to  provide  quick  and  efficient  collection  of 
new  types  of  information.  All  of  these  items  (reasons 
for  leaving,  demographic  information,  and  special 
answer)  were  combined  to  produce  a  two-sided  opti- 
cally-scannable  form.  The  opscan  format  was  chosen  to 
make  data  collection,  analysis,  and  display  as  efficient 
as  possible.  Computer  programs  were  written  to  scan 
the  questionnaires,  store  the  information  on  magnetic 
tape,  run  various  analyses,  and  print  summary  tables. 


Utilization 

This  management  information  system  was  designed 
specifically  for  utilization  by  the  Director,  Military  Per¬ 
sonnel  Policy  Division  (OP-13),  and  has  been  opera¬ 
tional  since  December  1 980.  Personnel  who  leave  the 
service  complete  the  questionnaire.  Questionnaires  are 
forwarded  to  the  Navy  Occupational  Development  and 
Analysis  Center  (NODAC)  in  Washington,  D.C.  Question¬ 
naires  are  analyzed  and  results  are  provided  quarterly. 
Figure  1  illustrates  the  flow  of  information  and  system 
procedure. 
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free-form  responses  were  analyzed  to  derive  the  most 
frequently  cited  reasons.  Based  on  the  “most  frequent" 
responses,  as  well  as  information  from  previous  sur¬ 
veys,  30  reasons  were  chosen.  Some  of  the  criteria 
used  in  the  selection  of  “reasons  for  leaving"  were 
whether  each  item  was: 

•  Within  the  power  of  the  Navy  to  change 

•  General  without  being  vague 

•  Different  from  other  reasons 

•  A  response  that  personnel  would  normally  give 
when  asked  "Why  are  you  separating?" 

A  questionnaire  has  been  developed  for  administra¬ 
tion  to  individuals  leaving  the  service.  Each  person  will 
be  aske£to  indicate  how  important  each  of  the  30  rea¬ 
sons  was  in  the  decision  to  separate.  To  facilitate  the 
analysis  of  subgroups,  various  demographic  items  are 
included,  such  as  rating,  pay-grade,  number  of  reen¬ 
listments,  and  marital  status.  A  special  answer  section 


is  included  to  provide  quick  and  efficient  collection  of 
new  types  of  information.  All  of  these  items  (reasons 
for  leaving,  demographic  information,  and  special 
answer)  were  combined  to  produce  a  two-sided  opti- 
cally-scannable  form.  The  opsean  format  was  chosen  to 
make  data  collection,  analysis,  and  display  as  efficient 
as  possible.  Computer  programs  were  written  to  scan 
the  questionnaires,  store  the  information  on  magnetic 
tape,  run  various  analyses,  and  print  summary  tables. 


Utilization 

This  management  information  system  was  designed 
specifically  for  utilization  by  the  Director.  Military  Per¬ 
sonnel  Policy  Division  (OP-13),  and  has  been  opera¬ 
tional  since  December  1980.  Personnel  who  leave  the 
service  complete  the  questionnaire.  Questionnaires  are 
forwarded  to  the  Navy  Occupational  Development  and 
Analysis  Center  (NODAC)  in  Washington.  D.C.  Question¬ 
naires  are  analyzed  and  results  are  provided  quarterly. 
Figure  1  illustrates  the  flow  of  information  and  system 
procedure. 
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Research  and  Development  Notes 

This  project  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC).  It  was 
sponsored  by  the  Deputy  Chief  of  Naval  Operations 
(Manpower,  Personnel  and  Training)  (OP-01),  using 
Operations  and  Maintenance.  Navy,  funds.  The  respon¬ 
sible  researcher  was  Dr.  W.  H.  Githens,  NPRDC  (Code 
12),  (714)  225-2408.  There  are  no  Technical  Reports 
published  on  this  work. 
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Impact 

The  automation  involved  in  this  system  has  reduced 
the  cost  of  providing  "reason  for  leaving"  information. 
This  information  is  used  to  develop  policies  and  proce¬ 
dures  related  to  retention.  The  impact  of  this  system 
will  be  seen  in  its  contribution  to  policies  that  result  in 
reduced  attrition. 
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EDUCATION  AND  TRAINING 


This  area  of  People-Related  RDT&E  is  defined  in  the  Department  of  Defense  as  the  “Development  of  education/ 
training  methods  and  media  for  managing,  designing,  and  evaluating  new-generetion  instructional  systems  for 
military  application." 

The  Navy  training  establishment  faces  major  challenges  in  trying  to  maintain  personnel  readiness  to  meet  opera¬ 
tional  demands  while  it  is  faced  with  economic  restrictions,  manpower  competition,  and  the  increasing  sophistication 
of  weapons.  New  technology  is  being  developed  to  focus  training  objectives  on  actual  job  requirements,  to  reduce 
training  costs  and  improve  efficiency,  to  improve  the  availability  of  training,  and  to  adapt  training  to  the  needs  of  the 
individual. 

The  projects  summarized  in  this  Annual  Report  are: 

•  Improving  the  Navy's  Computer- Managed  Instruction  System 

•  Computer-Assisted  Study  Management  in  a  Navy  " C "  School 

•  Enhancing  Computational  Capabilities 

•  Reducing  Front-End-Loaded  Training 
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IMPROVING  THE  NAVY'S  COMPUTER-MANAGED  INSTRUCTION  SYSTEM 


Need 

The  Navy  operates  a  large-scale  computer-managed 
instruction  (CMI)  system  that  handles  more  than  9,000 
students  daily.  Its  objective  is  to  train  students  in  the 
shortest  possible  period  of  time  to  specific  performance 
objectives.  An  analysis  of  the  system  identified  several 
major  problems:  inappropriate  use  of  instructors,  and 
inefficient  methods  for  scoring  CMI  tests,  were  two  of 
these  problems.  The  Chief  of  Naval  Education  and 
Training  (CNET)  and  the  Chief  of  Naval  Technical  Train¬ 
ing  (CNTECHTRA)  requested  research  and  development 
to  solve  these  problems. 

*5  Approach  and  Results 

The  approach  taken  to  providing  a  validated  role  for 
CMI  instructors  involved,  developing  a  role  model, 
comparing  the  role  model  and  actual  instructor  roles  in 
the  CMI  schools,  and  developing  a  training  program  for 
instructors  that  would  provide  the  skills  and  knowledge 
identified  in  the  theoretical  role  as  being  necessary. 

The  training  materials  were  provided  for  use  in  in- 
.  service  training  for  instructors.  Additionally,  the  role 
model  and  training  materials  were  provided  to  CNET  for 
inclusion  in  new  versions  of  the  curriculum  for  the  for¬ 
mal  Instructor  Training  course. 

Solution  to  the  problem  of  inefficient  testing  proce¬ 
dures  involved  two  thrusts.  One  thrust  developed  a 
calculator  like  device  to  be  used  by  students  instead  of 
paper  answer  sheets.  These  devices,  called  Test  Input 
Devices  (TIDs),  can  store  the  student  test  question 
answers  and  then  transmit  them  to  the  computer 
through  a  linking  device.  The  TIDs  have  been  proto¬ 
typed  and  evaluated  in  a  Basic  Electricity  and  Electron¬ 
ics  School  (BE&E).  In  the  evaluation,  the  TIDs  were 
found  to  eliminate  almost  all  mechanical  data  errors, 
and  to  save  about  10  percent  of  the  testing  time  com¬ 
pared  to  using  the  paper  answer  sheets. 

The  second  thrust  involved  an  analysis  of  the  CMI 
system  computer  architecture  and  operational  man¬ 
agement  structure,  to  identify  areas  that  contribute  to  a 
slow  system  response  time.  A  slow  response  time 
means  students  must  wait  to  receive  test  results.  This 
delay  is  costly.  The  analysis  involved  a  task  group  com¬ 
prised  of  representatives  of  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC),  CNET, 
CNTECHTRA.  the  Management  Information  and  Instruc¬ 
tional  System  Activity  (MIISA,  the  managing  computer 
organization),  the  Training  Analysis  and  Evaluation 
Group  (TAEG),  and  the  CMI  schools. 

As  a  result  of  the  analysis,  alternative  CMI  system 
designs  were  developed  that  would  provide  better  test¬ 
ing  capabilities  while  improving  system  response  time 
CNET  is  implementing  the  best  design  through  imme¬ 
diate  system  management  changes,  acquisition  and 
implementation  of  necessary  computer  equipment,  and 
development  of  a  distributed  computer  system  that 
retains  central  data  control. 


Utilization 

The  instructor  role  model  is  now  being  used  in  in- 
service  training  at  several  CNTECHTRA  CMI  schools.  It 
has  recently  been  distributed  to  all  CMI  schools.  The 
materials  have  been  provided  to  the  Navy  Instructional 
Program  Development  Center  that  is  responsible  for  the 
development  of  a  new  formal  Instructor  Training  cur¬ 
riculum.  This  is  the  first  time  a  CMI  instructor's  role  has 
been  developed  on  the  basis  of  what  the  instructor 
should  be  doing  as  part  of  the  instructional  system, 
rather  than  what  the  instructor  traditionally  does.  The 
technical  approach  taken  provided  a  new  means  for 
defining  "instructor"  as  new  delivery  systems  come 
into  existence. 

The  TIDs  were  evaluated  by  MIISA  for  proper  elec¬ 
tronic  operation,  and  in  the  BE&E  school  for  effective 
use  in  the  CMI  environment.  The  TIDs  were  evaluated 
by  students,  instructors,  and  researchers  as  easier  to 
use  and  more  effective  than  the  more  costly  paper 
answer  sheet  scoring  system.  CNET  and  TAEG  are  now 
conducting  an  economic  analysis  to  determine  the 
extent  of  system-wide  implementation.  The  Navy's  CMI 
system  is  in  the  forefront  of  large-scale  operational  CMI 
systems  but  can  still  benefit  from  the  incorporation  of 
new  computer  technology  developed  during  the  last  5-6 
years. 

The  results  of  all  of  these  efforts  were  communicated 
to  the  users  as  progress  was  made,  through  task  group 
meetings,  personal  contacts,  formal  briefings,  and 
training  program  drafts.  Formal  technical  reports  fol¬ 
lowed  later.  This  project  has  been  noted  for  its  close 
coordination  with  the  user  community  that  includes  the 
several  organizations  making  up  the  CMI  system. 


Impact 

The  results  of  this  work  will  be  used  by  CNET  instruc¬ 
tional  management  personnel  and  all  fifteen  CNTECH¬ 
TRA  schools  using  CMI.  Instructor  role  model  training 
will  improve  the  utilization  of  instructor  personnel. 
Results  to  date  have  demonstrated  improved  attitudes 
of  instructors.  As  more  courses  are  added  to  the  CMI 
system,  effective  use  of  the  available  staff  will  continue 
to  be  important. 

Replacement  of  the  paper  answering  system  with  the 
TIDs  will  reduce  test  scoring  errors  and  make  a  consid¬ 
erable  savings  in  system  costs.  The  cost  of  TIDs  is  pro¬ 
jected  to  be  amortized  in  about  two  years  by  elimination 
of  me  answer  sheets  and  the  associated  optical  mark 
readers. 

Toe  major  payoff  from  this  R&D  project  is  the 
upgraded  CMI  system  Implementation  is  planned  by 
CNET  over  the  next  two  years.  It  should  result  in  more 
effective  management  of  the  CMI  instructional  process. 
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REPRODUCED  at  government  EXPENSE 


Research  and  Development  Notes 


Program  dynamics  are: 


This  work  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  under  the  sponsor¬ 
ship  of  the  Deputy  Chief  of  Naval  Operations  (Man¬ 
power,  Personnel  and  Training)  (0P-01).  The  project 
identification  was  “Improving  Computer-Managed  In¬ 
struction  in  the  Navy,"  Z1 176-PN.01,  in  Program  Ele¬ 
ment  63720N,  Education  and  Training.  The  principal 
investigator  was  Dr.  N.  H.  Van  Matre,  NPRDC  (Code  1 4), 
(71 4)  225-7 1 22.  The  effort  extended  from  1 976  through 
1981.  The  project  was  previously  described  in  part  in 
the  Second  Annual  Report  on  Utilization  of  People- 
Related  Navy  RDT &£.  April  1979.  Selected  Reports  and 
Notes  issued  include  NPRDC  Special  Report  80-33, 
Computer  Managed  Instruction  in  the  Navy:  I  Research 
Background  and  Status,  September  1980;  NPRDC  Spe¬ 
cial  Report  81-26,  Upgraded  Navy  Computer  Managed 
Instruction:  Analysis  of  Requirements  for  and  Prelimi¬ 
nary  Instructional  System  Specifications.  September 
1981;  NPRDC  Technical  Note  80-10,  Theoretical  Defini¬ 
tion  of  Instructor  Role  in  Computer-Managed  Instruc¬ 
tion.  March  1980;  NPRDC  Special  Report  80-13, 
Requirements  and  Characteristics  of  a  Test  Answer 
Input  Device  for  the  Navy's  Computer  Managed  Instruc¬ 
tion  System.  March  1 980. 


COMPUTER- ASSISTED  STUDY  MANAGEMENT  IN  A  NAVY  "C"  SCHOOL 


Need 

The  Study  Management  System  (SMS)  is  a  learning 
management  procedure  developed  for  Navy  schools, 
featuring  self-study  of  existing  instructional  material. 
Student  interaction  with  the  subject  matter  is  ensured 
by  study  assignments,  numerous  study  questions,  and 
frequent  tests  with  feedback.  Both  instructor-managed 
and  computer-managed  forms  of  SMS  have  been  devel¬ 
oped  and  evaluated  in  college  classrooms  and  Navy 
Class  "A"  technical  schools.  The  computer-managed 
form  of  the  SMS  is  called  CAISMS  for  "Computer- 
Aided  Instruction  Study  Management  System." 

The  Chief  of  Naval  Education  and  Training  (CNET) 
foresaw  that  the  SMS  had  several  potential  benefits  in 
Navy  advanced  school.  Expectations  included  reduc¬ 
tions  of  1 5-20%  in  training  time  because  of  self-pacing; 
reductions  of  idle  time  waiting  for  courses  to  convene; 
particular  appropriateness  to  maintenance-oriented 
courses  requiring  use  of  technical  manuals  and  other 
job  aids;  and  easier  transferral  to  remote  site  use. 
Therefore  the  Navy  Personnel  Research  and  Develop¬ 
ment  Center  was  requested  to  investigate  the  feasibility 
of  SMS  in  a  Navy  advanced  maintenance  course. 


Approach  and  Results 

A  three-week  course  on  maintenance  of  a  fatho¬ 
meter  was  selected  by  (CNET)  as  the  course  for  experi¬ 
mentation.  During  the  first  phase  of  the  project  careful 
evaluation  was  done  in  the  current  course.  Student  and 
instructor  behaviors  known  to  be  related  to  learning 
and  memory  were  recorded. 

During  the  second  phase,  three  types  of  instructional 
materials  were  developed  to  support  the  implementa¬ 
tion  of  SMS:  materials  to  teach  the  use  of  the  technical 
manual  for  the  equipment;  lesson  assignments  with 
appropriate  study  questions  to  form  a  student  work¬ 
book;  and  a  written  testing  program.  All  the  test  ques¬ 
tions  were  designed  to  match  existing  learning  objec¬ 
tives.  About  twelve  man-months  of  effort  were  required 
for  conversion.  A  manual  describing  procedures  for 
converting  similar  courses  was  also  developed  for  use 
by  Navy  instructors. 

In  the  third  phase,  the  instructor-managed  SMS 
course  was  adapted  to  computer  management.  A  micro¬ 
computer-based  version  of  the  CAISMS  program  was 
used  to  manage  the  instruction  and  testing.  Conversion 
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of  the  SMS  version  of  the  course  to  CAISMS  took  less 
than  a  manmonth  of  effort.  A  manual  describing  opera¬ 
tion  and  use  of  the  microcomputer  version  of  the 
CAISMS  system  was  also  developed. 
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Comparisons  of  the  three  versions  of  the  course 
revealed  that  Navy  instructors  can  adapt  existing  group- 
paced  lecture-delivered  courses  to  the  SMS  system 
with  part-time  guidance  of  an  Educational  Specialist. 
Both  the  SMS  and  CAISMS  versions  of  the  course 
teach  the  intended  skills  and  knowledges.  In  addition, 
the  SMS  and  CAISMS  courses  were  more  efficient 
than  the  original  course.  The  amount  of  "hands-on" 
time  was  greater  for  SMS  and  CAISMS  students,  and 
they  received  much  more  instruction  and  practice  in 
using  the  technical  manual.  The  instructor  easily  man¬ 
aged  the  SMS  version  of  the  course. 

The  SMS  conversion  was  highly  cost-effective;  time 
savings  of  27%  were  achieved.  The  benefits  observed 
with  the  SMS  version  of  the  course  were  successfully 
transferred  to  the  CAISMS  version  of  the  course  and  a 
new  instructional  staff.  The  CAISMS  program  was  suc¬ 
cessfully  adapted  to  a  computer  system  affordable  and 
manageable  for  Navy  schools. 


Utilization 

The  SMS  versions  of  the  course  are  being  considered 
for  implementation  at  the  Fleet  Anti-Submarine  War¬ 
fare  Training  Center,  Pacific,  San  Diego.  CA,  and  at  the 
Naval  Submarine  School.  New  London,  CT.  In  addition, 
the  Chief  of  Naval  Education  and  Training  is  consider¬ 
ing  other  "C"  and  Fleet  courses  for  conversion  to  the 
SMS.  and  is  beginning  acquisition  of  computer  systems 
Suitable  for  CAISMS. 

The  course  examined  in  this  effort  is  representative 
of  many  of  the  approximately  3000  Navy  advanced 
courses  that  are  offered  each  year.  On  the  basis  of  the 
results  observed  in  this  effort,  previous  CAISMS  eval¬ 
uations.  and  many  other  studies  concerned  with  self- 
paced  instruction,  it  is  probable  that  the  benefits  of 
SMS  or  CAISMS  will  transfer  widely. 
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Impact 

The  conversion  of  the  original  three-week  course  to 
SMS  took  about  ore  man-year  of  effort;  similiar  con¬ 
version  times  for  similiar  course  can  be  anticipated.  The 
conversion  must  be  done  by  subject-matter  experts, 
which  in  most  cases  would  be  the  instructors)  assigned 
to  the  course.  These  initial  conversion  costs  are  offset 
by  the  benefits  that  are  associated  with  self-paced, 
flexible-entry  course.  If  the  time  savings  of  27%  were 
maintained  for,  say,  100  students,  then  about  two  stud¬ 
ent  man  years  will  have  been  saved.  Futher.  reduced 
wait-time  for  courses,  increased  student  throughput, 
and  greater  availability  of  courses  for  personnel 
assigned  to  ships,  may  be  realized. 

The  conversion  from  SMS  to  CAISMS  required  less 
than  a  month  of  clerical  effort.  The  major  costs  involved 
in  converting  an  SMS  course  to  CAISMS,  however,  are 
the  initial  computer  purchase  and  subsequent  mainte¬ 
nance  costs.  The  computer  configuration  which  is  most 
appropriate  depends  on  a  number  of  variables.  The 
microcomputer  used  in  this  project  was  purchased  for 
about  $10,000,  has  maintenance  costs  of  about  $500 
per  year,  and  may  last  up  to  five  years  depending  on 
amount  of  use.  Less  expensive  computers  can  also  run 
the  CAISMS  system,  but  they  may  have  to  be  replaced 
more  frequently  An  estimate  for  yearly  computer  costs 
is  somewhere  between  $200  and  $500,  multiplied  by 
the  average  or  peak  student  load.  However,  computeri¬ 
zation  of  the  course  may  not  result  in  benefits  in  addi¬ 
tion  to  those  which  result  from  conversion  to  SMS.  In 
this  project,  the  CAISMS  students  showed  small 
increases  in  test  scores,  but  also  took  slightly  longer  to 
complete  the  course.  In  general,  no  substantial  effec¬ 
tiveness  or  efficiency  gains  or  losses  were  measurable 
relative  to  the  SMS  version. 

Computerization  could  be  cost-effective  in  instructor 
utilization,  in  the  SMS  course,  instructors  hand-scored 
and  provided  feedback  for  tests;  this  took  nearly  three 
hours  per  student.  This  is  a  clerical  task  which  the 
CAISMS  computer  could  perform.  If  it  can  be  managed, 
the  "saved"  instructor  time  could  be  used  to  teach  addi¬ 
tional  courses,  or  convert  other  cou  ses  to  SMS  or 
CAISMS. 
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AT  GOVERNMENT  EXPENSE 


Research  and  Development  Notes 

This  effort  was  performed  by  the  Navy  Personnel 
Research  and  Development  Center  in  1980  and  1981. 
It  was  sponsored  by  the  Deputy  Chief  of  Naval  Opera¬ 
tions  (Manpower,  Personnel  and  Training)  (OP-01)  as 
“Improved  Effectiveness  in  Course  Design,  Delivery  and 
Evaluation,"  Z1175-PN.05,  in  Program  Element 
63720N,  Education  and  Training.  Previous  Exploratory 
Development  work  had  beeen  accomplished  in  Pro¬ 
gram  Element  62763N,  Personnel  and  Training  Tech¬ 
nology.  The  Defense  Advanced  Research  Projects 

Program  dynamics  are. 


Agency  also  provided  funding.  The  responsible  re¬ 
searcher  was  Dr.  W.  E.  Montague,  NPRDC  (Code  1 3). 
(714)  225-7121.  There  are  eight  documents  describing 
the  work  and  the  results  of  this  project.  The  more 
recent  are  NPRDC  TR  82-33,  Development.  Test  and 
Evaluation  of  the  Computer-Assisted  Study  Manage¬ 
ment  System  in  a  Navy  "C"  School:  Summary  Report. 
February  1 982;  Special  Report  (SR)  82-().  User’s  Guide 
for  Microcomputer  Applications  of  CAISMS.  in  press; 
and  SR  82-().  Procedures  for  Adapting  Group-Paced 
Instruction  Materials,  in  press. 


ENHANCING  COMPUTATIONAL  CAPABILITIES 


Need 

The  sophistication  of  military  equipment  is  increas¬ 
ing;  training  budgets  are  not  increasing  at  the  same 
rate.  To  assure  cost  effective  training,  skills  and  knowl¬ 
edge  essential  to  successful  job  performance  in  the 
fleet  and  subordinate  skills  necessary  to  master  essen 
tial  skills  must  be  identified  Conversely,  skills  not 
required  for  successful  performance  must  be  identified 
and  removed  from  entrance  standards  and  course 
objectives. 


Over  23,000  recruits  who  enter  Navy  Class  "A" 
schools  every  year  are  trained  in  electronics  mainte¬ 
nance.  Before  these  recruits  enter  "A"  school,  how¬ 
ever,  they  must  successfully  complete  training  in  the 
fundamentals  of  electronics  at  one  of  the  Basic  Electric¬ 
ity  and  Electronics  (BE/E)  schools. 

Although  the  selection  requirements  for  the  electron¬ 
ics  schools  are  more  stringent  than  for  most  areas  of 
Navy  technical  training,  many  students  are  not  pre¬ 
pared  to  begin  the  BE/E  school  curriculum  The  BE/E 


K'  ’ 

h.  *• 


L  .. 


course  and  all  subsequent  electronics  courses  use 
mathematics  to  express  relations  in  electronics  sys¬ 
tems.  Deficiencies  in  mathematics  appear  to  be  prime 
contributors  to  unsatisfactory  performance  in  electron¬ 
ics.  Hence,  a  need  exists  to  specify  objectively  the 
mathematics  skills  required  to  become  a  competent 
Navy  electronics  maintenance  technician,  to  assess 
current  skill  levels,  and  to  provide  remedial  training  for 
the  skills  found  deficient. 

The  original  intent  of  this  work  unit  was  to  specify  the 
mathematics  skills  required  to  become  a  competent 
Navy  electronics  maintenance  technician.  However,  at 
the  request  of  the  Chief  of  Navy  Technical  Training 
(CNTECHTRA),  the  effort  was  expanded  to  include  the 
construction  of  diagnostic  tests  to  assess  a  student's 
ability  to  perform  identified  BE/E  mathematics  pre¬ 
requisites,  and  the  development  of  remedial  mathemat¬ 
ics  curriculum  modules  keyed  to  the  diagnostic  tests. 


Approach  and  Results 

The  first  step  was  to  identify  mathematics  require¬ 
ments.  A  survey  form  was  developed  for  administration 
to  instructors  at  the  BE/E  and  electronics  "A"  and  "C" 
schools.  Seventy  skills  were  identified  from  standard 
texts  on  mathematics  for  electronics.  The  purpose  of 
the  survey  was  to  identify  levels  of  importance  of  these 
skills  to  school  success.  The  number  of  skills  judged  to 
affect  performance  at  the  BE/E  school  was  21,  of 
which  six  were  considered  to  be  prerequisites  and  were 
not  reviewed  or  taught  in  the  BE/E  School,  two  were 
reviewed  in  the  school,  and  13  were  taught.  Across  all 
"A"  schools  surveyed,  twenty-five  percent  of  the  math 
skills  were  considered  to  be  prerequisites,  44  percent 
as  reviewed,  and  31  percent  as  taught.  Across  all  "C” 
schools  surveyed,  fifty-two  percent  of  the  math  skills 
were  considered  to  be  prerequisite,  29  percent  as 
reviewed,  and  19  percent  as  taught.  Next,  mathematics 
requirements  for  job  performance  were  surveyed.  Very 
little  math  use  was  found  at  the  job  station;  however, 
over  50  percent  of  the  respondents  claimed  that  their 
math  knowledge  helped  them  on  the  job.  The  data 
resulting  from  the  surveys  provided  the  basis  for 
determining  the  prerequisite  mathematics  requirements 
of  the  BE/E  school. 

Once  identified,  these  prerequisites  were  broken 
down  into  subskill  components  which  formed  the  basis 
for  three  diagnostic  tests,  each  to  be  administered  just 
prior  to  the  point  in  the  BE/E  curriculum  where 
mathematics  skills  were  required.  To  accompany  the 
diagnostic  tests,  mathematics  learning  modules  were 
prepared.  Each  module  was  keyed  to  specific  math 
items  appearing  on  the  diagnostic  tests.  The  students 
performing  poorly  were  remediated  on  the  specific 
skills  with  which  they  were  experiencing  difficulty, 
thereby  keeping  remedial  training  time  to  a  minimum. 

Two  hundred  students  enrolled  in  the  BE/E  school. 
Naval  Training  Center,  San  Diego  participated  in  the 
pilot  effort.  Preliminary  results  indicated  that,  when 
compared  with  the  traditional  BE/E  math  modules,  the 
experimental  mathematics  training  made  a  significant 
difference  in  electronics  performance  in  some  areas 
Overall,  however,  no  significant  differences  were  found 
in  BE/E  course  performance  and  time  to  complete  the 
course  between  students  using  either  the  traditional  or 
experimental  modules.  Investigation  to  determine  why 
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there  was  little  difference  revealed  that  the  mathemat¬ 
ics  skills  of  the  experimental  group  were  higher  than 
normal  so  that  the  experimental  modules  could  not 
show  their  effect. 


Utilization 

Efforts  are  currently  underway  to  obtain  a  group  of 
students  whose  math  aptitude  is  judged  by  BE/E  per¬ 
sonnel  to  be  more  representative  of  the  overall  BE/E 
population.  However,  based  on  the  positive  direction  of 
findings  with  the  high  aptitude  mathematics  group,  the 
Chief  of  Naval  Technical  Training  is  preparing  all  four 
BE/E  schools  to  convert  to  the  experimental  mathemat¬ 
ics  curriculum. 


Impact 

Proper  training  in  mathematics  skills  required  for 
learning  electronics  should  serve  to  reduce  the  attrition 
and  setback  rate  at  the  BE/E  schools,  and  improve  fleet 
performance.  By  diagnosing  a  student's  mathematics 
difficulties  through  testing  and  then  providing  specially- 
designed  materials,  the  amount  of  remedial  work  re¬ 
quired  will  be  reduced.  This  will  decrease  total  training 
costs  and  preserve  a  more  positive  student  self-concept. 


Research  and  Development  Notes 

The  Navy  Personnel  Research  and  Development  Cen¬ 
ter  performed  this  work  during  the  period  1977  to 
1982.  The  project  was  "Enhancement  of  Computa¬ 
tional  Capabilities",  and  it  was  funded  in  Exploratory 
Development  Program  Element  62763N,  Personnel  and 
Training  Technology.  Dr.  M.  S.  Baker,  NPRDC  (Code  15), 
(714)  225-6803  was  the  responsible  researcher.  Five 
technical  reports  have  been  published,  including  NPRDC 
TR  82-3,  Mathematics  Course  Requirements  and  Per¬ 
formance  Levels  in  the  Navy’s  Basic  Electricity  and 
Electronics  Schools.  October  1981,  and  TR  82-5, 
Mathematics  Requirements  of  Electronics  Ratings  in 
the  Job  Environment.  October  1 981 . 

Program  dynamics  are: 


APPLICATION 

Remedial  mathematic*  training 
at  the  basic  electricity  and 
electronics  schools; 

•  San  Diego,  NTC 

•  Great  Lakes.  NTC 

•  Orlando,  NAS 

•  Memphis,  NAS 
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REDUCING  FRONT-END-LOADED  TRAINING 


Need 

The  Navy's  current  approach  to  technical  training  is 
"front-end-loaded";  that  is.  technical  training  is  pro¬ 
vided  immediately  after  recruit  training.  As  a  result, 
heavy  investments  are  made  in  "A"  school  and  "C" 
school  training  while  high  uncertainty  exists  with 
respect  to  return  on  that  investment.  Thirty  to  forty  per¬ 
cent  of  the  individuals  who  begin  technical  training 
never  reach  a  related  shipboard  job.  Many  who  com¬ 
plete  training  ashore  have  difficulty  in  adapting  to  life 
aboard  ship  and  therefore  perform  at  marginal  levels. 
The  end  result  is  an  intolerably  low.  return  on  first  term 
enlisted  technical  training. 

One  means  of  reducing  the  costs  of  early  training  is 
to  apply^b  performance  aid  (JPA)  technology.  Numer¬ 
ous  studies  of  JPA  applications  over  the  past  10  years 
have  improved  maintenance  and  made  more  effective 
use  of  available  personnel.  In  1976,  the  Director,  Naval 
Education  and  Training  (OP-099)  initiated  this  project  to 
test  and  evaluate  the  application  of  job  performance 
aids.  As  the  project  has  developed,  the  scope  has  been 
significantly  broadened.  It  now  includes  the  develop¬ 
ment  and  test  of  an  innovative  personnel  system.  The 
Deputy  Chief  of  Naval  Operations  (Manpower,  Person¬ 
nel  and  Training)  (OP-OI )  has  continued  support  for  the 
project,  and  the  Commander-in-Chief,  U  S.  Pacific  Fleet 
has  endorsed  it  favorably. 

The  objective  of  this  work  is  to  develop,  implement, 
and  field  test  an  experimental  parsonne)  system  which 
will: 

Q  Provide  early  job  assignment  on-board  ship 
directly  after  recruit  training,  using  JPAs. 

O  Eliminate  front-end-training  by  distributing  shore- 
based  and  shipboard  training. 

©  Individualize  career  advancement  paths. 

©  Improve  utilization  of  "lesser 'aptitude"  person¬ 
nel,  i.e.  those  currently  considered  "non-school 
eligible." 

©  Improve  retention. 


Approach  and  Results 

The  approach  taken  is  based  on  the  premise  that  cur¬ 
rent  Navy  personnel  problems  have  their  roots  in  the 
personnel  system  itself  and  therefore  can  best  be 
addressed  through  a  total  "systems  approach".  There¬ 
fore,  an  Integrated  Personnel  Systems  Approach  (IPSA) 
was  devised.  It  considers  training  design.  JPA  design, 
career  structures,  advancement  channels,  incentives 
and  the  quality  and  quantity  of  personnel  resources 
available.  Coupled  with  a  cost  tradeoff  model,  IPSA  was 
employer!  to  evolve  a  new  personnel  system  concept 
entitled  Enlisted  Personnel  Individualized  Career  Sys¬ 
tem  or  "EPICS".  The  NATO  Seasparrow  Surface  Missile 
System  (NSSMS)  is  the  vehicle  for  the  test  and  evalua¬ 
tion  of  EPICS 


EPICS  products  to  date  include: 

•  Digital  system  troubleshooting  aids. 

•  EPICS  documentation  materials. 

—  Recruiting  pamphlet  for  potential  EPICS 
personnel. 

—  Sailor’s  Handbook  for  EPICS  sailors  that  de¬ 
scribes  the  EPICS  career  model. 

—  Orientation  Pamphlet  for  briefing  middle  and 
top  level  leaders. 

—  EPICS  Administration  Guide  for  the  EPICS 
shipboard  administrator  to  facilitate  manage¬ 
ment  of  the  EPICS  program. 

•  EPICS  job  design  for  NSSMS. 

•  NSSMS  instructional  packages: 

—  Self-teaching  exportable  modules. 

—  Shorebased  training  packages. 

—  Training  management  support  materials  and 
career  counseling  guidelines. 

•  NSSMS  fully  procedufalized  job  performance  aids 

(84). 

•  NSSMS  partially  proceduralized  job  performance 

aids  (114). 

•  NSSMS  troubleshooting  job  performance  aids. 


Utilization 

Most  of  the  components  required  for  the  implemen¬ 
tation  of  EPICS  in  the  fleet  for  the  test  and  evaluation 
have  been  completed.  An  OP-01  steering  group  was 
established  to  insure  EPICS  is  implemented  to  the 
fullest  extent  possible.  Briefings  and  orientations  were 
provided  at  all  levels,  from  OP-01  to  the  work  center 
supervisors  aboard  each  participating  ship.  AH  EPICS 
sailors  have  been  installed  onboard  30  DD-963s  and  4 
CVs.  The  first  shorebased  training  course  has  been  in 
operation  since  June  1981.  The  curriculum  develop¬ 
ment  of  the  second  shorebased  school  has  been  com¬ 
pleted  and  the  school  began  operation  in  October  1981. 
Data  for  the  test  and  evaluation  have  been  and  are 
continuing  to  be  collected.  Analysis  of  the  test  and  eval¬ 
uation  findings  will  provide  detailed  data  for  use  in  judg¬ 
ing  whether  or  not  Navy-wide  implementation  should 
be  undertaken. 

Upon  completion  of  the  test  and  evaluation,  an  IPSA 
development  guide  will  be  compiled.  This  guide  will  be 
a  handbook  that  delineates  the  guidelines,  algorithms, 
cost  benefit  and  cost  effectiveness  trade-offs,  and  base¬ 
line  data  developed  over  the  life  of  the  project.  Potential 
users  of  the  IPSA  handbook  are  agencies  and  individu- 


22 


i 


l 


\  • 


ats  who  are  responsible  for  or  are  developing  a  JPA- 
based  integrated  personnel  and  training  system.  IPSA 
can  be  use  to  evaluate  alternative  system  designs  and 
to  determine  the  role  of  JPAs  early  in  the  system  acqui¬ 
sition  process.  Generalization  to  other  systems  and 
ratings  requires  the  acquisition  of  JPAs  and  training 
materials  for  those  ratings  selected,  and  support 
documentation. 

Impact 

Other  than  the  more  long-term  benefits  noted  as 
objectives  of  the  project  several  products  will  have 
direct  and  immediate  benefit  to  the  operating  Navy. 
These  products  include: 

•  JPAs  on  the  NSSMS  fire  control  and  launcher 
systems  which  are  to  be  used  by  "A"  school 

-  graduates  to  facilitate  early  performance. 

•  Innovative  digital  troubleshooting  aids  (State 
Tables)  which  are  being  validated  and  produced 
for  general  fleet  distribution. 

•  A  five-module  self-administered  ship  indoctrina¬ 
tion  program  adaptable  to  general  ship  use. 
Currently,  three  PACFLT  DO-963s  are  using  the 
series  in  that  fashion.  The  Commander-in-Chief, 


U.S.  Atlantic  Fleet  and  Chief  of  Naval  Education 
and  Training  are  evaluating  the  shipboard  indoc¬ 
trination  package  lor  Navy-wide  use. 

•  Instructional  aids  for  NSSMS  system  operation 
and  maintenance,  which  may  be  used  for  all 
Combat  Systems  Department  personnel  for  basic 
operational  and  maintenance  instruction  on  the 
NSSMS  system. 

•  Troubleshooting  JPAs  for  the  launcher  system 
which  can  be  used  as  learning  aids  during  "C" 
schools. 


Research  and  Development  Notes 

The  Navy  Personnel  Research  and  Development  Cen¬ 
ter  has  been  the  performing  laboratory  for  this  project. 
The  title  is  “Enlisted  Personnel  Individualized  Career 
System  (EPICS)",  Z0828-PN,  in  Program  Element 
63720N,  Education  and  Training.  The  responsible 
researcher  is  Dr.  R.  E.  Blanchard,  NPRDC  (Code  17), 
(714)  225-6617.  The  project  started  in  1976  and  will 
complete  in  1985.  Technical  Reports  and  Notes  are 
available.  Important  among  these  is  NPRDC  TN  80-14. 
Integrated  Personnel  Systems  Approach  (IPSA):  Enlisted 
Personnel  Individualized  Career  System  (E PICS)  Model. 
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APPLICATIONS 

POTENTIAL 

Job  performance  aids  integrated 

After  4 -year?  ten,  general!  re  to 

• 

with  training,  job  design,  career 

other  systems  and  career  paths 

I 

path  design,  and  technical  data. 

on  board  other  Navy  platform. 

Demonstration  using  NATO 
Seasperrow  Surface  Missile 

System  on  board  all  DD-963* 
and  4  CVs. 

1* 

6.3  AOVANCED  DEVELOPMENT 


Design,  develop,  test,  end  evaluate  job 
performance  aid-based  ‘migrated 
personnel  system. 


•w 
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mmm  factors  engineering 


in  the  Department  of  Defense,  human  factors  engineering  is  defined  as  the  "Development  of  improved  methods 
and  technologies  for  the  analysis,  design,  and  evaluation  of  equipment/systems  for  safer  and  more  efficient  opera¬ 
tion  and  maintenance." 

The  Navy  needs  equipment  and  support  systems  designed  in  such  a  way  that  people  can  do  their  jobs  faster,  more 
accurately,  and  more  safely  when  they  operate,  maintain,  or  employ  equipment.  The  Navy’s  research  and  develop¬ 
ment  program  in  human  factors  engineering  is  involved  in  all  systems,  from  their  initial  formulation  to  test  engineer¬ 
ing  specialists  in  various  development  agencies  and  contractor  firms. 

Projects  in  this  category  include: 

•  Designing  Software  Documentation 

•  Designing  for  Maintainers 

•  Energy  Management  in  Navy  Family  Housing 

•  Matching  Men  and  Machines 
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DESIGNING  SOFTWARE  DOCUMENTATION 


Need 

Sensor  and  weapon  systems  performance  is  critically 
dependent  on  computers  whose  cost  and  reliability  are 
determined  primarily  by  their  software.  Maintenance 
and  re-fitting  of  that  software  are  usually  performed  by 
individuals  other  than  the  original  designers,  and  are 
therefore  heavily  dependent  on  the  software  documen¬ 
tation  available.  Many  techniques  have  been  adopted 
for  thejepresentation  of  software  but  there  has  been 
little  evaluation  of  their  general  utility.  The  specification 
of  the  most  helpful  types  of  current  and  proposed  doc¬ 
umentation  techniques  to  aid  programmers  would 
sharply  reduce  the  life-cycle  costs  of  computer-based 
equipments.  Also,  when  documentation  is  suitable, 
there  is  less  need  to  re-initiate  the  design  and  devel¬ 
opment  process  and  thus  there  is  a  considerable  saving 
in  time  to  get  the  re-programmed  system  back  on  line. 


Approach  and  Results 

This  research  program  started  with  the  selection  of 
several  techniques  for  representing  software  and  order¬ 
ing  the  events  in  the  programs.  Three  forms  of  repre¬ 
sentation  —  ideograms,  constrained  language,  and  nar¬ 
rative  statements  —  were  combined  with  three  ordering 
techniques  —  sequential,  branching,  or  distributed  — to 
make  nine  documentation  formats  These  formats  were 
evaluated  in  programs  that  solved  complex  engineering 
functions  (computation  of  a  rocket  trajectory),  manipu¬ 
lated  large  data  bases  (an  inventory  control  problem),  or 
combined  both  problem  types  (an  airport  traffic  control 
program). 

Programmer  performance  was  measured  in  several 
tasks:  understanding  the  organization  of  a  software 
program,  location  and  correction  of  errors  in  software 
code,  comprehension  of  code,  and  modification  of  code. 
In  each  task,  measures  were  taken  of  completion  times, 
error  occurrences,  and  frequency  of  use  of  various 
commands  and  computer  resources. 

Significantly  superior  performance  resulted  when  a 
sequential  ordering  technique  was  coupled  with  the 
use  of  constrained  language.  This  type  of  format  is  the 


basis  of  one  currently-used  technique  known  as  pro¬ 
gram  design  language  (POL). 


Utilization 

The  documentation  technique  that  proved  to  be  most 
effective  under  a  wide  range  of  conditions  and  mea¬ 
surements  has  been  generally  adopted  by  NASA- 
Goddard,  IBM  Federal  Systems  Division,  General 
Electric  Company  and  the  International  Telephone  and 
Telegraph  Corporation.  The  Marine  Corps  Tactical  Sup¬ 
port  System  Activity  has  accepted  it  for  test  report  plan 
documentation  and  the  technique  has  been  incorpo¬ 
rated  into  a  programming  course  sponsored  by  a  pro¬ 
fessional  organization,  the  Institute  of  Electrical  and 
Electronic  Engineers.  In  addition,  several  Navy  Labora¬ 
tories  and  Centers  (e.g.,  the  Naval  Research  Laboratory 
and  the  Pacific  Missile  Test  Center)  are  exploring  the 
potential  operational  software  packages  as  system 
changes  are  introduced. 


Impact 

Software  development  costs  for  computer-based  sys¬ 
tems  both  in  military  and  commercial  use  now  com¬ 
mand  90%  of  total  costs.  Current  dollar  investment  in 
software  development  of  naval  systems  is  difficult  to 
estimate  but  for  FY  73  it  was  calculated  to  be  between 
$1.0  -  1.3  billion,  which  was  about  4%  of  the  Navy 
budget  that  year.  The  costs  are  estimated  to  have 
tripled  by  FY77.  Many  techniques  are  in  use  for  docu¬ 
menting  software  and  PDL  is  estimated  to  be  employed 
m  about  1 0%  -  30%  of  those  programs.  Good  documen¬ 
tation  permits  about  a  25%  -  40%  reduction  in  life-cycle 
costs  since  those  data  are  in  use  continuously.  Esti¬ 
mates  of  improvement  that  could  be  projected  by  ex¬ 
tending  the  use  of  techniques  with  the  characteristics 
of  PDL  range  from  1  - 10%.  Based  on  these  estimates  a 
conservative  estimate  of  cost  savings  that  would  have 
been  possible  in  FY  77  ranges  between  $5  -  50  million 
per  year. 
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Research  and  Development  Notes 

This  project  was  performed  under  the  sponsorship  of 
the  Chief  of  Naval  Research,  by  the  General  Electric 
Company,  Information  Systems  Programs,  1755  Jeffer¬ 
son  Davis  Highway,  Arlington,  Virginia.  It  is  one  of  a 
cluster  focused  on  the  enhancement  of  human-com¬ 
puter  interaction.  The  project  was  funded  in  Program 
Element  61153N,  Defense  Research  Sciences,  Sub¬ 
element  42,  Behavioral  and  Social  Sciences,  project 
RR042-09,  Engineering  Psychology.  The  responsible 
researcher  was  Dr.  J.  J.  O'Hare,  ONR  (Code  442),  (202) 

Program  dynamics  are: 


696-4502.  General  Electric  Company's  principal  inves¬ 
tigator  was  Dr.  E.  Kruesi.  The  project  was  initiated  in 
1979  and  will  be  completed  in  1982.  There  are  several 
technical  reports  available,  dealing  with  the  effects  of 
symbology  and  spatial  arrangement  on  the  comprehen¬ 
sion  of  software  specifications  (TR-80-388200-2. 
October  1980),  in  a  debugging  task  fTR-80-388200-4! 
August  1981),  in  a  coding  task  (TR-8 1-388200-3! 
February  1981).  and  in  a  modification  task  (TR-81- 
388200-5),  December  1981). 


APPLICATION 

Specify  i  Mended  documented  on  formet 
Ettebliih  e  reference  be**  for  eeefuedon  of  propeced  format* 
Provide  *  be*i>  for  co*t  ett!  motet  for  toftwvt  modification* 


DESIGNING  FOR  MAINTAINERS 


Need 

The  problem  of  maintaining  airborne  systems  has 
grown  considerably  over  the  past  two  decades.  With 
the  increased  size  and  ever-growing  complexity  of 
modern  aviation  weapon  systems,  the  costs  of  mainte¬ 
nance  are  increasingly  a  disproportionate  portion  of  the 
Department  of  Defense  budget.  Furthermore,  as  many 
as  1  /3  of  all  military  personnel  may  be  detailed  exclu¬ 
sively  to  maintenance  and  support  functions  However, 
traditional  solutions  to  this  problem  have  not  worked— 


a  typical  squadron  today  may  have  available  only  about 
60%  of  its  aircraft  for  full  operational  use.  For  example, 
an  analysis  of  F-14  3-M  data  reveals  that  because  of 
excessive  mean  elapsed  maintenance  times  (EMT),  a 
$2.3B  excess  inventory  of  F-14's  is  needed  to  maintain 
a  prescribed,  mission-capable  force;  and  that  main- 
tainer  errors  (e.g..  diagnostic  false  alarms,  or 
maintenance-produced  damage)  alone  result  in  a  re¬ 
quirement  for  an  additional  1.23  maintainers  per  air¬ 
craft  merely  to  recover  from  maintainer  performance 
errors. 
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Clearly,  when  it  comes  to  operational  readiness,  reli¬ 
ability  is  and  always  has  been  a  key  issue.  But  because 
of  the  acceleration  in  the  complexities  of  subsystems 
which  characterize  modern  aircraft,  even  if  overall  sys¬ 
tem  reliability  could  manage  to  sustain  or  jmprove 
present  day  levels,  measured  maintainability  (M)  would 
continue  to  decline.  While  such  technological  advances 
as  built-in-test  {BIT}  and  modularization  certainly  show 
promise,  early  predictions  for  their  success  were  quite 
probably  overly  optimistic  {e.g.,  BIT  reliability  for  the  F- 
14  is  disappointing).  Furthermore,  the  dwindling  supply 
and  increasing  costs  of  a  qualified  population  of 
organizational-level  maintainers  serve  to  exacerbate 
the  problem. 

The  essence  of  M  is  fast,  safe,  efficient  repair,  and  at 
its  heart  ^fe  people-related  variables  — factors  such  as 


diagnostic  behavior,  decision/  cognitive  complexities, 
accessibility,  psychomotor  coordination,  anthropometric 
matchups,  transfer  of  training,  and  systematization. 
These  human  factors  must  be  comprehensively  and 
systematically  examined.  8nd  the  performance-based 
discoveries  which  research  provides  must  be  elegantly 
intermeshed.  The  Commander,  Naval  Air  Systems 
Command  has  sponsored  a  technology  development 
program  to  meet  this  need. 


Approach  and  Results 

The  approach  involves  three  thrusts  as  depicted  in 
the  diagram  below.  Each  thrust  contributes  its  own 
product,  and  each  also  supports  the  other  thrusts. 


THRUST 


PRODUCTS 


DATA  BASE  ANALYSIS/SYNTHESIS 

•  Naval  Safety  Center  Accident/Mijhap  Analysis 

•  F-14  3-M  Analysis 

•  P-4  3-M  Analysis 

•  F-18  3  M.  FLAG  T&E  Analysis 


Quantitative  Picture  of 
Meintainer  Performance 


INTERIM  ISOLATIONS/SOLUTIONS 

•  Extract  Design  Feature  Problems 

•  Identify  Solutions 

•  Validate  Solutions 


MAINTENANCE  ACTION  MODEL 

•  Develop  Cost-Effective  Simulation  of  Design 
Impact  on: 

•  Maintenance  Performance 

•  Aircraft  Availability 

•  Manpower  Savings 

•  Validate/Update  Model 


Experimental  Checklist  Development/ 
Validation 

•  Laboratory/Simulation  Data 

•  Fleet  Data 

•  ECP  Development 


Design  Aids 

•  Interactive  Models 

•  MILSTD’s 


The  first  thrust  is  an  analysis  and  synthesis  of  The  F-14  and  F-4  analyses  began  with  an  identifica- 

organizational-level  maintenance  performance  data.  tion  of  subsystems  for  which  chronological  analyses  of 

Several  platforms  are  involved,  atthough  there  is  a  con-  3-M  reports  show  a  high  incidence  of  maintainer  diag- 

centration  on  tactical  air,  particularly  on  F-4,  F-14,  and  nostic  (i.e.,  false  alarm)  errors,  and  a  high  incidence  of 

F-18  data.  damage  by  remove-repair  or  replacement.  Subse¬ 

quently,  a  per  form  a  nee -based  model  of  the  design  fea- 
The  Safety  Center  analysis  involved  the  culling  of  a  tures  associated  with  these  maintainer  errors  was 

1977-1980  bank  of  accident  and  mishap  data,  in  order  developed.  It  shows  that  a  given  investment  in  design 

to  extract  inferences  about  the  relationships  between  modifications  leads  to  a  quantifiable  improvement  in 

equipment  design  features,  standard  maintenance  pro-  maintenance  performance  (decreased  EMT).  and  a  pre- 

cedures,  and  the  safety  and  efficiency  of  maintenance.  dictable  gain  in  aircraft  availability,  coupled  with  a 

Oespite  the  methodological  problems  associated  with  decrease  in  necessary  manpower.  Furthermore,  the 

examining  a  data  base  not  designed  for  this  type  of  F-14  and  F-4  analyses  led  to  a  procedure  by  which 

analysis,  several  key  human  factors  problem  areas  experimental,  organizational-level  data  can  be  collected 

have  been  isolated,  and  recommendations  for  improve-  in  order  to  validate  the  rationale  behind  forthcoming 

ment  are  being  formulated.  engineering  change  proposals  (ECPs). 
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The  F-1 8  analysis  arose  from  an  application  of  state- 
of-the-art  human  factors  principles  aimed  at  satisfying 
the  stringent  M  criteria  which  were  contractually  speci¬ 
fied  for  that  aircraft.  As  3-M  data  become  available  for 
this  newly-deployed  weapon  system,  several  hundred 
human  factors  and  maintainability  features  which  the 
F-1 8's  design  incorporated  are  being  monitored  and  the 
data  analyzed. 

Utilization 

The  ultimate  objective  of  this  effort  is  the  develop¬ 
ment  of  a  set  of  design  aids,  including  a  maintenance 
action  model,  and  appropriate  Military  Standards 
(MILSTDs)  and  Military  Specifications  (MILSPECs).  Cur¬ 
rently,  to  validate  the  approach  and  the  interim  solu¬ 
tions  derived  thus  far,  retrofit  recommendations  are 
being  formulated  and  tested.  One  example  is  the  devel¬ 
opment  of  an  engineering  change  proposal  (ECP) 
directed  at  improving  the  maintainability  of  the  F-14's 
fire  detection  sensing  element.  An  analysis  of  3-M  data 
revealed  that  the  average  elapsed  maintenance  time 
per  sensing  element  maintenance  action  was  nearly 
four  hours.  Investigations  showed  that  the  maintainers 
themselves  had  developed  simplified  procedures  rather 
than  follow  current  procedure  standards.  Although  this 
solution  eliminates  some  portion  of  the  excessive  EMT, 
the  procedure  is  itself  time-consuming,  and  is  often 
impractical  in  close-quarters  environments.  The  recom¬ 
mendation  of  the  Designing  for  Maintainers  project 
was  to  provide  a  relatively  simple  adapter  to  circumvent 
any  access  requirement  external  to  the  engine  nacelle, 
at  an  estimated  EMT  savings  of  two  to  three  hours  (i.e., 
50-75%)  per  maintenance  action.  Validation  of  this 
estimated  impact  is  in  progress,  as  is  a  simulated  eval¬ 
uation  of  the  overall  cost-effect  impact  on  the  F-1 4 
community.  This  procedure  by  which  data  analysis 
leads  to  problem  isolation,  from  which  proposed  solu¬ 
tions  are  identified  and  validation  then  follows,  provides 
immediate  fleet  solutions.  It  also  provides  the  techno¬ 
logical  base  for  the  ultimate  development  of  design 
aids.  These  aids  are  the  tools  which  will  allow  for  the 
incorporation  of  HF  principles  at  the  earliest  conceptual 
phases  of  aircraft  evolution. 

Impact 

Maintainer  errors  can  be  reduced  substantially  if 
design  is  effectively  blended  with  emphases  on  training 
and  job  performance  aid  development.  Under  study 
conditions,  maintainer  errors  were  reduced  by  as  much 
as  97%.  This  would  translate  to  nearly  25%  improve¬ 
ment  in  the  utilization  of  available  manpower  resources. 


Research  and  Development  Notes 

The  performing  activity  for  this  project  is  the  Naval 
Air  Development  Center  (NADC),  Warminster,  PA.  The 


project  title  is  "Design  for  Maintainers"  and  it  is  funded 
in  Program  Element  62757N.  Human  Factors  and  Sim¬ 
ulation  Technology.  The  principal  investigator  is  LT  D.  K. 
McBride.  MSC.  USNR,  NADC  (Code  6021),  (215)  441- 
2561 .  Among  many  of  the  publications  from  thisproject 
have  been  NADC -792 18-60,  Design  lor  Maintainers: 
Final  Report.  1982;  and  NADC  HFTM  81-06.  Human 
Factors  in  Design  for  Maintainability,  1 982. 

Program  dynamics  are: 
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ENERGY  MANAGEMENT  IN  NAVY  FAMILY  HOUSING 


Need 

Declining  energy  supplies  and  skyrocketing  energy 
costs  have  made  it  essential  that  the  Navy  emphasize 
conservation.  In  Navy  family  housing,  substantial  re¬ 
ductions  in  utility  consumption  have  been  made  through 
technological  and  structural  modifications.  These  "pas¬ 
sive"  conservation  approaches  require  little  or  no  active 
participation  by  residents.  However,  a  large  component 
of  reduction  in  energy  consumption  depends  on  the 
individual  consumer.  As  housing  facilities  are  im¬ 
proved,  the  behavior  of  the  consumer  becomes  a  more 
important  source  of  savings  through  energy  conserva¬ 
tion.  The  challenge  is  to  develop  methods  for  gaining 
the  active.participation  of  energy  users  in  the  conserva¬ 
tion  proet&s.  Recognizing  this  need,  the  Navy  Facilities 
Engineering  Command  (NAVFACENGCOM)  requested 
that  the  Navy  Personnel  Research  and  Development 
Center  (NPRDC)  develop  practical  approaches  to  bring 
about  active  resident  participation  in  utility  conservation. 


Approach  and  Results 

Studies  of  energy  conservation  in  the  civilian  popula¬ 
tion  have  demonstrated  the  applicability  of  behavioral 
strategies  to  the  reduction  of  utility  consumption.  How¬ 
ever,  families  in  Navy  housing  are  influenced  by  factors 
not  common  in  the  general  population.  Although  sav¬ 
ings  are  possible,  these  factors  must  be  considered  in 
designing  energy  conservation  programs.  The  factors 
include  the  historical  nature  of  housing  benefits  and 
their  administration,  the  policy  of  including  utilities  in 
rent,  master-metering  of  housing  complexes  so  that 
individual  consumption  rates  are  not  available,  and 
regulations  preventing  the  use  of  incentives  for  conser¬ 
vation  achievements. 

The  first  phase  of  the  effort  was  to  describe  the 
conservation-related  attitudes  and  practices  of  Navy 
family  housing  residents  and  to  evaluate  a  conservation 
program  for  master-metered  housing  sites.  Results 
indicated  that  substantial  energy  savings  are  possible, 
that  personal  communication  is  important  for  obtaining 
resident  involvement,  and  that  changes  in  consumption 
tend  to  endure  after  cessation  of  the  program. 

Based  on  this  background,  practical  guidelines  were 
defined  for  energy  conservation  programs  in  Navy  fam¬ 
ily  housing  sites.  Principles  identified  as  contributing  to 
the  effectiveness  of  the  design  and  implementation  of 
past  tests  of  energy  conservation  programs  in  Navy 
family  housing  were: 

•  Communicate  personally  with  residents 

•  Vary  program  intensity 

•  Provide  conservation  information 

•  Set  conservation  goals 


•  Express  Navy  concern 

•  Demonstrate  housing  support 

•  Involve  the  entire  family 

•  Develop  and  maintain  proconservation  attitudes 

•  Provide  feedback  on  usage 

•  Commend  conservation  efforts 


A  manual  was  written  with  sections  corresponding  to 
each  of  the  principles.  For  each  principle,  there  is  a  brief 
explanation,  a  description  of  how  it  can  be  applied,  and 
an  appendix  providing  specific  information,  materials, 
and  examples.  The  manual  provides  housing  adminis¬ 
trators  with  an  efficient,  step-by-step  approach  to 
designing  and  conducting  a  local  energy  conservation 
program  over  an  extended  time  period. 


Utilization 

The  manual.  Energy  Management  for  Navy  Family 
Housing:  A  Manual  for  Voluntary  Residential  Energy 
Conservation,  was  delivered  to  NAVFACENGCOM  in 
October  1981.  NAVFACENGCOM  requested  NPRDC  to 
develop  a  second  package,  to  provide  documentation 
and  training  materials  on  the  techniques  available  for 
housing  managers  to  monitor  and  control  utility  costs. 
Work  is  in  progress  to  develop  material  on  procedures 
for  conducting  energy  audits  of  existing  facilities  to 
identify  areas  for  structural  or  hardware  improvement 
projects,  relationships  between  energy  consumption 
ard  housing  maintenance,  and  performance  character¬ 
istics  and  new  technologies  available  for  inclusion  in 
acquisition  of  new  hardware  or  new  construction.  A 
menual  will  be  prepared  to  distribute  these  new  mate¬ 
rials  to  housing  managers. 

Training  programs  on  managerial  techniques  and 
educational  approaches  are  being  developed  for  the 
Family  Housing  Management  Institute.  These  will  be 
presented  and  evaluated  at  the  Institute  during  1982. 


Impact 

Training  of  housing  managers  in  utility  conservation 
is  expected  to  curb  utility  cost  overruns  which  divert 
funds  from  housing  maintenance. 

The  savings  to  be  realized  from  utility  conservation 
have  been  estimated  to  range  from  15-50%.  A  realistic 
estimate  is  20-25%.  Projections  are  that  energy  costs 
will  double  in  five  years.  The  cost  avoidance  to  be 
achieved  by  energy  management  will  thus  continue  to 
grow  in  importance. 
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Research  and  Development  Notes 

This  project  is  being  performed  by  the  Navy  Person- 
net  Research  and  Development  Center  under  the  spon¬ 
sorship  of  the  Commander.  Naval  Facilities  Engineering 
Command.  The  responsible  researcher  is  Dr.  B.  Feher, 
NPRDC  (Code  16),  (714)  225-2191.  In  addition  to  the 


manuals  and  materials  already  described,  NPRDC  Tech¬ 
nical  Report  82-20,  Energy-related  Attitudes  of  Navy 
Family  Housing  Residents  is  available,  as  are  several 
papers  which  have  been  delivered  by  NPRDC  project 
members  at  various  meetings.  The  Energy  Manage¬ 
ment  Manual  previously  described  is  NPRDC  Technical 
Report  82-22. 


Program  dynamics  are: 


APPLICATION 

Entrgy  conservation  module*  at 
Family  Homing  Management  Inttitute 

Nation-wide  dittribution  of  energy  management 
manual  to  300  housing  sites 


REIMBURSABLE  FUNDS 

r  Development  of  a  manual  to  guida  housing  managers'1 
in  designing  and  conducting  long  term  energy 
^conservation  programs  among  family  housing  residents. 

Development  of  energy  management  training 
modules  for  use  at  die  Family  Housing 
Management  Institute. 


reimbursable  funds 

Assessment  of  attitudes  and  practices  of  Navy  family 
housing  tenants. 

Field  experiments  testing  strategies  for  including 
^voluntary  utility  conservation  by  family  housing 
residents. 


MATCHING  MEN  AND  MACHINES 


Need 


The  circumstance  is  rare,  if  it  ever  occurs,  in  which  a 
weapon  system  functions  independently  from  any 
human  involvement.  The  range  of  human  involvement 
will  vary  ,  in  some  cases,  only  a  few  knobs  and  dials  may 
have  to  be  manipulated  and  read  at  appropriate  times; 
in  other  cases,  operator  involvement  is  of  such  strong 
and  continuing  nature  that  the  human  is  truly  a  major 
subsystem.  Certainly,  for  the  pilots  and  NFOs  in  high 
performance  Navy  aircraft,  the  latter  situation  pertains. 


However,  cockpits  represent  space,  weight,  and  other 
factors  which  must  be  considered  carefully  in  design. 
Further,  many  displays,  mechanisms,  and  furnishings 
are  built  into  the  cockpit  for  operational  and  life  support 
purposes;  the  occupant  must  be  able  to  fit,  touch,  see. 
move,  hear,  pull,  push,  turn,  and  otherwise  perform 
specific  tasks  at  various  points  in  the  mission  profile.  As 
more  and  more  sophistication  is  incorporated  into  sys¬ 
tems  while  the  performance  requirements  for  the  sys¬ 
tem  continue  to  escalate,  the  capability  of  the  system  to 
accommodate  the  human  operator  and  the  operator  to 
accomplish  the  tasks  set  by  the  system,  become  more 
and  more  a  matter  of  concern. 
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Many  aspects  of  the  hardware  design  process  are 
based  on  “best  guess"  or  subjective  decisions  by 
designers  and  engineering  managers.  Typical  of  these 
decisions  have  been  the  physical  grouping  and  location 
of  displays  and  controls,  the  allocation  of  crewstation 
panel  space,  and  the  selection  of  labels  and  abbrevia¬ 
tions.  However,  inappropriate  control  and  display 
arrangements  create  serious  reach  problems  for  opera¬ 
tors.  Therefore,  under  the  sponsorship  of  the  Deputy 
Chief  of  Naval  Operations  (Air  Warfare)  (OP-05),  and 
the  Commander,  Naval  Air  Systems  Command,  work 
was  undertaken  to  provide  a  better  basis  for  decisions 
in  the  design  process. 

Approach  and  Results 

Several  computer  routines  addressing  specific  design- 
related  problems  areas  were  developed.  All  routines 
are  interactive — that  is,  the  user  can  work  with  the 
computer  to  use  and  adjust  the  models  in  order  to  test 
various^lternative  designs.  The  collective  group,  identi¬ 
fied  as  Interactive  Design  Support  Models  (IDSM),  con¬ 
sists  of: 

9  Panel  space  allocation  (CUBITS) 

9  Labeling  and  abbreviation  (ABBREV) 

9  Crewstation  Assessment  of  Reach  (CAR) 

9  Automated  Operational  Sequence  Diagram 
(AOSD) 

9  Functional  Grouping  of  Controls  /Displays 
(GROUP) 


CUBITS 

Current  design  procedures  for  the  determination  and 
allocation  of  panel  space  for  controls  and  displays  of  the 
various  systems  integrated  into  aircrew  stations  do  not 
involve  adequate  consideration  of  the  purpose  and 
characteristics  of  those  devices.  CUBITS  developed  a 
systematic  and  objective  computational  method  for  the 
allocation  of  panel  areas  in  aircrew  workspaces.  The 
procedure  is  based  on  the  criticality  of  the  control  or 
display  to  crew  safety  and  mission  effectiveness;  the 


frequency  of  utilization;  and  the  amount  of  information 
which  is  conveyed  by  the  operator  to  the  system 
through  control  actuation  or  which  is  conveyed  to  the 
operator  by  the  display  presentation. 

ABBREV 

In  crewstation  design  the  various  displays  and  con¬ 
trols  are  usually  arranged  on  panels  as  functional 
groups  serving  a  common  system  or  subsystem  pur¬ 
pose.  The  display  and  control  groups  are  designed  and 
installed  to  provide  the  operator  with  all  the  information 
and  control  required  for  the  employment  of  the  various 
integrated  system  capabilities.  In  many  cases,  display 
panels  that  provide  the  status  of  a  system  are  located 
separately  from  their  control  panels,  due  to  highly 
limited  space  in  the  crewstation  and  to  operator  reach 
limitations.  As  a  result,  clear,  concise,  and  consistent 
labeling  to  allow  rapid  and  accurate  association  with 
these  dislocated  displays  and  controls  is  essential  for 
effective  aircrew  performance.  The  objective  of  this 
effort  is  to  produce  a  computer  model  for  the  systematic 
selection  and  abbreviation  of  the  labeling  for  crew¬ 
station  controls  and  displays. 

CAR 

In  the  past,  design  standards  required  that  a  crewsta¬ 
tion  be  compatible  with  the  3rd  and  98th  percentile 
aviator.  In  attempting  to  comply  with  this  requirement, 
the  crewstation  designer  was  forced  to  assume  perfect 
correlation  among  body  measurements.  An  analysis  of 
the  assumed  'constant  percentile'  aviators  vs.  the  actual 
aviator  population  indicated  the  need  to  design  and 
evaluate  crewstations  according  to  the  actual  mea¬ 
surements  of  the  intended  operators.  The  evaluative 
method  must  be  sufficiently  rapid  to  minimize  expendi¬ 
tures  and  must  be  convenient  enough  to  ensure  that 
the  analyst  will  employ  the  technique.  To  this  end,  the 
Crewstation  Assessment  of  Reach  (CAR)  Model  was 
developed.  The  purpose  of  the  CAR  Model  is  to  estab¬ 
lish  the  actual  percentage  of  naval  aviators  that  can  be 
accommodated  in  the  critical  areas  of  a  given  crewsta¬ 
tion  geometric  configuration.  The  model  examines  hand 
and  leg  control  positions,  seat  movement  to  establish 
over-the-nose  vision,  and  head  clearance  for  a  repre¬ 
sentative  sample  of  the  naval  aviator  population.  Figure 
1  illustrates. 


AOSO 

An  Operational  Sequence  Diagram  (OSD)  is  a  sophis¬ 
ticated  graphic  elaboration  of  task  analysis  procedures. 
OSDs  allow  a  sequential,  time-based  description  of  a 
system's  operation.  They  display  system  element  func¬ 
tional  relations,  information  and  material  flow,  sequen¬ 
tial  distribution  .of  operation,and  identify  subsystem 
control  and  information  inputs  and  outputs.  Although 
graphic  OSDs  are  of  tremendous  value  in  system 
design,  the  tedium  and  cost  associated  with  modifying 
and  regraphing  diagrams  substantially  reduces  their 
utility.  The  objective  of  the  Automated  OSD  effort  is  to 
provide^  computerized  interactive  graphics  method  for 
creation  and  updating  of  OSDs  with  minimum  manual 
redrafting. 


GROUP 

The  placement  of  controls  and  displays  relative  to  one 
another  within  a  crewstation  can  be  carried  out  in 
numerous  alternative  ways.  In  general,  the  best  ar¬ 
rangements  will  be  those  which  minimize  learning  time 
and  which  produce  fewest  errors  during  operation.  The 
GROUP  procedure  uses  the  inherent  meaning  in  the 
words  for  a  device  label  as  a  description  of  the  function 
of  that  device.  Grouping  is  carried  out  by  placing  more 
closely  together  those  devices  which  have  more  of  their 
functions  in  common.  This  is  accomplished  by  quantify¬ 
ing  the  extent  of  likeness  among  devices,  and  by  statis¬ 
tical  grouping  procedures. 


Utilization  and  Payoff 

CUBITS  has  been  applied  on  a  number  of  aviation 
systems,  including  the  S-3A,  SH-3H,  PROTEUS,  LAMPS 
MK-II,  AIDS  (Advanced  Integrated  Display  System), 
LINEBACKER,  IACS  (Integrated  Avionics  Control  Sys¬ 
tem).  DAIS  (Digital  Avionics  Instrumentation  System), 
and  LAMPS  MK-lll.  ABBREV  has  been  applied  to  the 
S-3A,  SH-3H,  PROTEUS,  LINEBACKER,  and  IACS.  CAR 
has  been  applied  to  the  F-18  and  LAMPS  MK-lll.  As  a 
result  of  these  applications,  benefits  such  as  the  follow¬ 
ing  have  been  realized: 


CUBITS 

D  On  the  PROTEUS  program,  there  was  a  90% 
reduction  in  time  spent  to  accomplish  typical 
maintenance  tasks  as  compared  with  the  original 
panel  design. 


•  For  SH-3H  TACAN,  operator  functions  were 
grouped  logically  and  the  random  intermix  of  dis¬ 
similar  control  devices  was  eliminated. 

•  Recommendations  for  improvement  of  space 
allocation  and  layout  for  all  displays  and  controls 
were  produced  for  the  F-18  program. 

•  Application  to  the  LINEBACKER  program  resulted 
in  the  introduction  of  appropriate  push-button 
switching  to  improve  system  operability  and 
reduce  error. 

•  A  reallocation  of  the  panel  space  to  permit  use  of 
improved  input  switching  concepts  resulted  from 
application  to  IACS. 

•  In  DAIS,  CUBIT  resulted  in  recommendations  for 
optimum  utilization  of  control/display  panel  space 
and  served  as  a  guideline  for  comparative  evalua¬ 
tions  and  competitive  designs. 

ABBREV 

•  For  LAMPS,  ABBREV  resulted  in  a  20%  reduction 
in  both  operator  "time-to-learn"  and  "time-to- 
operate." 

•  In  the  F- IB  program,  application  resulted  in  con¬ 
tractor  changes  to  the  system. 

•  Application  to  the  LINEBACKER  program  reduced 
both  the  likelihood  of  operator  error  and  the 
length  of  training  required  to  reach  operational 
proficiency. 

CAR 

•  On  the  LAMPS  program.  CAR  disclosed  six  reach 
problems  in  cockpit  geometry.  These  problems 
were  corrected  during  later  development. 

•  Application  to  the  F-18  resulted  in  the  identifica¬ 
tion  of  several  serious  deficiencies  in  aircrew 
reach  to  critical  controls  in  the  cockpit.  The  con¬ 
trols  were  relocated,  resulting  in  a  60%  increase 
in  accommodation  of  aircrew  population. 

The  value  of  these  models  has  been  recognized  by 
the  aviation  industry,  as  well  as  the  National  Aeronau¬ 
tics  and  Space  Administration.  Users  now  include  the 
NASA  Johnson  Space  Center,  McDonnell  Douglas  Cor¬ 
poration,  Northrop,  IBM  Corporation,  Clark  Equipment 
Company,  and  Sikorsky. 
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Research  and  Development  Notes 

The  performing  activity  for  this  project  is  the  Naval 
Air  Development  Center  (NADC),  Warminister,  PA.  Early 
portions  of  the  effort  were  accomplished  with  funding 
from  Program  Element  62757N,  Human  Factors  and 
Simulation  Technology.  Later  effort  was  funded  in  proj¬ 
ect  W1189-PN,  “Computer  Assisted  Methods  for 
Human  Factors  Engineering  Design  and  Evaluation," 
Program  Element  63701 N,  Human  Factors  Engineering 
Technology  Development.  Among  the  principal  investi¬ 
gators  have  been  CAPT  P.  R.  Chatelier,  MSC,  USN  (202) 
695-9777;  CDR  P.  M.  Curran,  MSC,  USN  (202)  696- 
4713;  -Dr.  J.  Hobson  (215)  441-2561;  CDR  C.  W. 
Hutchins,  MSC,  USN  (202)  696-7443;  CDR  N.  E.  Lane, 
MSC,  US&U(305)  646-5317;  Mr.  P.  Unto  (215)  441- 
2561 ;  and  Dr.  N.  Warner  (21 5  441  -251 6.  Among  many 


Technical  Reports  and  other  documentation  are  Dyna- 
mation,  Inc..  Report  No.  7905-1,  Design  Procedure  for 
an  Information  Transfer  Method  {"CUBITS")  of  Allocat¬ 
ing  Panel  Area  (or  Aircrew  Station  Controls  and  Dis¬ 
plays.  May  1 979;  Analytics,  Inc.,  TR-1 398,  The  Human 
Operator  Simulator.  Vol II:  HOS  Users  Guide  (revised/. 
May  1979;  Analytics,  Inc.,  TR-1 400,  The  Crewsta'ion 
Assessment  of  Reach  Program:  Recommendations  for 
Modification  and  Enhancement.  September  1979; 
Analytics,  Inc.,  TR-1 400.06B,  CAR-II:  A  Revised  Model 
for  Crevvstalion  Assessment  of  Reach.  June  1 980; 
Analytics,  Inc.  TR-1400.08B,  Adapting  the  Human 
Operator  Simulator  Model  to  Large  System  Evaluation 
(draft),  January  1981;  and  Analytics,  Inc.,  TR-1 400.1 2 A, 
Analysis  of  the  Decision  Aid  Evaluation  Capability  of  the 
HOS/HOPROC  System.  April  1981 . 


Program  dynamics  are: 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


SIMULATION  AND  TRAINING  DEVICES 


In  the  Department  of  Defense,  this  area  of  People-Related  RDT&E  involves  the  "Development  of  cost  effective 
trairfifig  equipment  and  technology  that  produce  the  needed  performance  for  operation  and  maintenance  of  military 
systems.'' 

The  Navy  needs  training  devices  and  simulators  for  several  purposes:  to  improve  readiness  through  realistic 
exercise;  to  reduce  training  costs;  to  increase  safety  during  practice  of  dangerous  activities;  and  to  reduce  the 
destructive  impact  of  training  activities  on  the  environment. 

Projects  in  this  category  include: 

•  Advanced  Fire  Fighting  Training 

•  Anti-Armor  Missile  Flight  Training 

•  Applications  of  Computer  Voice  Technology 
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ADVANCED  FIRE  FIGHTING  TRAINING 


Need 

Few  occurrences  can  rival  the  impact  on  any  sea¬ 
soned  mariner  of  a  catastrophic  fire  at  sea.  An  uncon¬ 
trolled  fire  on  board  a  Navy  ship  threatens  life  and 
hazards  equipment  and  the  ship  itself.  Thrust  into  the 
situation  of  a  fire  at  sea,  even  relatively  inexperienced 
personnel  rise  to  the  occasion  and  perform  heroically  in 
fighting  the  blaze.  However,  routine  mistakes  are  just 
as  commonplace  as  heroics.  Utilization  of  improper 
extinguishing  agents;  lack  of  coordination  between  fire 
fighting  teams,  which  negates  effective  response;  and 
failure  to  isolate  the  source  of  additional  flammables; 
are  illustrative. 

All  Navy  recruits  are  exposed  to  the  rudiments  of 
Navy,  fire  fighting  procedures  while  going  through 
recruit  training.  In  addition,  advanced  fire  fighting  train¬ 
ing  mustbe  provided  for  all  personnel  assigned  to  fire 
fighting  parties,  engineering  personnel,  personnel 
assigned  to  flight  or  hanger  deck  duties,  and  personnel 
who  routinely  handle  flammable  or  explosive  materials. 
However,  Fleet  Fire  Fighting  Schools  cannot  meet  these 
training  requirements  due  to  a  lack  of  modern  training 
media  and  instructional  delivery  systems.  There  are 
also  other  problems,  such  as  need  for  faster  turn¬ 
around  time  between  training  events,  and  a  need  to 
reduce  safety  hazards  to  personnel.  Overall,  there  is  a 
lack  of  capability  to  present  fire  situations  safely  in  real¬ 
istic  scenarios  and  ship-board  environments  for  the 
quantity  of  personnel  requiring  training. 

Further,  existing  firefighting  trainers  are  not  consis¬ 
tent  with  the  need  to  minimize  the  introduction  of  un¬ 
necessary  pollutants  into  the  atmospheres.  Trends  in 
regulations  on  air  and  water  pollution  necessitate  the 
development  of  a  nontoxic,  nonpolluting  fire  fighting 
training  environment  which  is  consistent  with  the  cri¬ 
teria  for  clean  air  and  water  established  by  the  Environ¬ 
mental  Protection  Agency,  state  and  local  ordinances. 

To  meet  these  needs,  the  Director,  Naval  Education 


and  Training  (OP-099)  established  a  requiremenl  for 
development  of  improved  fire  fighting  training  systems. 
The  requirement  has  continued  under  the  sponsorship 
of  the  Deputy  Chief  of  Naval  Operations  (Manpower, 
Personnel  and  Training)  (0P-01),  with  strong  support¬ 
ing  interest  by  the  Director,  Submarine  Manpower  and 
Training  Requirements  Division  (OP-29),  the  Director, 
Surface  Warfare  Manpower  and  Training  Requirements 
Division  (OP-39),  and  the  Chief  of  Naval  Education  and 
Training. 

Approach  and  Results 

Exploratory  development  by  the  Naval  Training 
Equipment  Center  identified  a  viable  alternative  to 
existing  smoke  abatement  techniques.  A  new  approach 
to  the  generation  of  fires  in  a  controlled  environment 
used  a  clean  burning  gaseous  fuel  (propane)  in  con¬ 
junction  with  a  logic  control  unit  and  extinguishment 
sensors.  Sensors  detect  the  quantity  and  location  of  an 
applied  extinguishing  agent  and  send  signals  via  the 
logic  unit  to  control  the  flow  of  propane  gas.  The  degree 
of  realism  is  predetermined  by  sensor  arrangement  and 
responsiveness  to  the  various  actions  of  sweeping  the 
extinguishing  agent  over  the  sensors. 

Based  on  acceptable  feasibility  tests,  a  fire  fighting 
trainer  model  was  built  and  tested.  The  model  featured 
two  simulated  oil  fires  and  a  simulated  electrical  panel 
fire.  In  a  separate  effort,  the  generation  of  "training 
smoke”  was  also  investigated.  Successful  testing  of  the 
model  paved  the  way  for  engineering  development  of  a 
prototype  fire  fighter  trainer. 

Phase  I  of  the  prototype  fire  fighter  trainer  (device 
19F1)  consisting  of  five  fires,  was  constructed  at  the 
Fleet  Training  Center,  Norfolk,  Virginia,  and  accepted  by 
the  Navy  in  February  1 981 .  The  1 9F1  is  a  bi-level  struc¬ 
ture  containing  eight  compartments  representative  of  a 
shipboard  environment.  At  one  side  of  the  top  level  is 
an  instructor  control  room.  Figure  1  illustrates. 


Figure  1 
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Mock-ups  of  shipboard  equipment  are  installed 
(Figure  2)  and  obstructions  normally  encountered 
onboard  ship  are  included  for  realism.  Gas  burners  are 
installed  so  as  to  produce  a  realistic  fire. 


Payoff 

The  Advanced  Fire  Fighter  Trainer,  Device  19F1 .  pro¬ 
vides  a  realistic  shipboard  setting  for  training  Navy  per¬ 
sonnel  in  shipboard  fire  fighting  techniques  in  a  safely 


Figure  2 


Students  enter  the  19F1  from  the  roof  top  "main 
deck”  and  proceed  below  in  the  same  manner  as  they 
would  on  a  ship.  Extinguishing  equipment  is  all  stan¬ 
dard  shipboard  type.  Nonflammable,  minimally  pollut¬ 
ing,  smoke  is  generated  in  modified  commercial  equip¬ 
ment  and  piped  to  compartments  to  provide  realistic  for 
the  trainees.  Substitute  but  realistic  fire  fighting  agents 
are  used  to  reduce  material  costs  and  to  simplify  the 
treatment  of  water  and  air  to  remove  pollutants. 


Utilization 

Government  testing  of  Device  19F1  for  the  first  five 
fires  met  all  contractual  requirements.  Realism,  train¬ 
ing  cues  and  fire  performance  were  favorable.  After 
some  changes,  environmental  impact  constraints  have, 
for  the  most  part,  been  satisfied.  The  trainer  with  seven 
fires  is  scheduled  to  undergo  operational  suitability  and 
training  effectiveness  tests  in  1982.  As  a  result  of  the 
development  and  construction  of  Device  1 9F1 ,  the  Navy 
has  the  capability  to  simulate  effectively,  in  a  controlled 
environment,  a  variety  of  training  fires  that  are  per¬ 
ceived  by  the  student  as  realistic  to  the  situation,  class 
and  type  of  fire,  and  type  and  method  of  extinguish¬ 
ment.  Utilization  of  the  propane  burner  and  a  sensor- 
driven  computer  system  has  fulfilled  the  requirement 
for  rapid  ignition,  shutdown  and  restart,  as  well  as  dis¬ 
playing  the  characteristics  of  flame  flare-up,  spreading, 
and  reflash  if  not  properly  extinguished.  For  training 
realism,  environmentally  acceptable  smoke  can  be 
introduced  when  required. 


controlled  environment.  EPA  constraints  have  been  sat¬ 
isfied  and  little  or  no  pollution  control  equipment  is 
required  to  meet  effluent  constraints.  The  technique  of 
simulating  the  various  classes  of  fires  with  propane 
burners  affords  significant  cost  reductions  through  con¬ 
trolled  use  of  gas  fuel  in  lieu  of  combustibles  and  using 
cost-effective  substitute  extinguishing  agents.  Elimina¬ 
tion  of  combustibles  also  speeds  up  turnaround  time  for 
training  and  reduces  preparation  and  clean-up  costs. 

Device  19F1  was  designed  to  be  expanded  to  include 
a  complete  scenario  of  fire  types  representative  of  ship¬ 
board  situations.  Utilized  by  highly  trained  instructor 
personnel.  Device  1 9F1  offers  the  potential  for  a  highly 
efficient,  well  instrumented  and  controlled  test  platform 
to  evaluate  new  fire  fighting  equipment  and  techniques. 
Already,  data  developed  during  the  initial  evaluation  of 
Device  19F1  are  being  utilized  to  aid  in  the  develop¬ 
ment  of  a  new  generation  of  Navy  fire  fighting  trainers. 

The  technology  can  be  applied  to  training  for  Navy 
civilian  fire  fighters  (for  example,  at  Naval  Air  Stations 
and  in  shipyards)  by  configuring  the  specific  fire  situa¬ 
tion  with  the  necessary  sensors  and  burners.  Similar 
application  can  be  made  to  the  training  requirements  of 
the  firemen  of  any  municipality. 

Research  and  Development  Notes 

The  Naval  Training  Equipment  Center  is  responsible 
for  development  of  the  prototype  Navy  Fire  Fighting 
Trainer,  the  19F1.  The  project  is  titled  "Advanced  Fire 


40 


Fighting  Simulator".  S0790-PN,  in  Program  Element 
64703N,  Training  Devices  Prototype  Development, 
sponsored  by  the  Deputy  Chief  of  Naval  Operations 
(Manpower,  Personnel  and  Training)  (OP-01).  A  pre¬ 
vious  Exploratory  Development  project  was  funded  in 
Program  Element  (PE)  62757N,  Human  Factors  and 
Simulation  Technology.  A  new  exploratory  development 
funded  by  PE  62757N  was  initiated  in  FY82  to  investi¬ 
gate  unique  requirements  for  effectively  fighting  fires  in 
submarines  and  to  develop  improved  fire  fighting  train¬ 
ing  equipment.  The  responsible  researchers  are  Mr.  I*. 
M.  Sciascia,  NTEC  (PD301).  (305)  646-5031  and  Mr.  £. 
Swiatosz,  NTEC  (N731),  (305)  646-5964.  The  principal 


investigator  is  Mr.  R.  Ambrozy,  Advanced  Technology 
Systems,  1 701  Pollitt  Drive.  P.O.  Box  950,  Fairlawn,  NJ 
07410,  (201)  794-0200.  The  project  started  in  1976 
and  will  complete  in  1984.  A  previous  description  was 
provided  in  the  2nd  Annual  Report  of  the  Utilization  of 
Navy  People-Related  RDT&E.  Additional  information 
may  be  found  in  an  article  entitled  "New  Developments 
in  Navy  Fire  Fighter  Trainers";  Proceedings.  1st  Inter- 
service/ Industry  Training  Equipment  Conference: 
(NAVTRAEQUIPCEN IH-31 6),  27-29  November  1 979.  p. 
217-227. 

Program  dynamics  are: 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


ANTI-ARMOR  MISSILE  FLIGHT  TRAINING 


The  U.S.  Army  and  Marine  Corps  now  have  in  their 
weapon  inventories  an  anti-armor  missile  known  as 
ORAGON.  It  is  a  guided  missile  system  designed  for 
either  offensive  or  defensive  operations  within  a  1 .000 
meter  range.  DRAGON  can  be  carried  by  a  man.  Fired 
from  a  bipod  launcher,  the  missile  is  tracked  optically  by 
a  gunner  and  guided  automatically  to  the  target  by  elec¬ 
trical  signals  transmitted  via  a  wire  link.  The  gunner 
fires  the  DRAGON  missile  by  depressing  the  safety  and 
squeezing  the  trigger.  Then  the  gunner  must  keep  the 
sight  on  the  target  However,  immediately  upon  squeez¬ 
ing  off  the  round,  the  gunner  is  deafened  by  the  sound 
of  the  igniting  rocket  motor;  forced  backward  by  a  recoil 
force;  just  as  suddenly  finds  himself  rolling  forward 
with  the  rocket  tube  and  sight  tilted  downward  due  to 
weight  loss  as  the  rocket  exits  the  tube.  At  the  same 
time,  he  j£sitting  in  a  cloud  of  smoke  which  marks  him 
as  a  target  and  obscures  his  view  through  the  sight  as 
he  tries  to  pick  up  the  target.  The  gunner  must  quickly 
overcome  all  these  distractions  because  he  has  a 
$5,000  missile  under  his  control  heading  for  a  target. 
He  must  regain  his  sight  picture  and  smoothly  track  the 
target,  ignoring  the  missile  which  may  be  prominent  in 
his  sight  picture.  To  score  a  hit.  the  gunner  must  over¬ 
come  many  perturbations  that  can  spoil  his  aim.  A  well 
qualified,  steady,  highly  trained  individual  is  required  as 
the  gunner.  However,  the  use  of  live  weapons  for  quali¬ 
fying  gunners  would  be  inordinately  expensive;  also 
employing  a  training  system  which  does  not  incorpo¬ 
rate  the  perturbations  described  would  not  provide 
qualified  gunners. 

A  study  conducted  for  the  Army  had  indicated  serious 
shortcomings  in  the  capability  of  the  Army  to  train  indi¬ 
viduals  effectively  in  the  use  and  employment  of  the 
DRAGON  anti-armor  missile  weapon  system.  During 
the  final  stages  of  development  of  the  Universal  Infan¬ 
try  Weapons  Trainer  (UIWT)  by  the  Naval  Training 


Equipment  Center  (NAVTRAEQUIPCENX  a  proposal  was 
made  to  the  Army  Project  Manager.  Training  Devices 
(PM  TRADE)  to  initiate  a  follow-on  effort  to  develop  a 
DRAGON  trainer.  The  Army  concurred  in  the  require¬ 
ment  and  sponsored  development.  Subsequently,  the 
Marine  Corps  also  stated  a  requirement  and  provided 
funding. 

Approach  and  Results 

Personnel  from  NAVTRAEQUIPCEN  visited  Army  and 
Marine  Corps  sites  conducting  DRAGON  training  and 
brought  "expert''  personnel  to  Orlando  to  participate  in 
an  initial  effort  to  define  precisely  what  the  nature  of 
training  had  to  be.  The  determination  was  made  that  an 
effective  DRAGON  trainer  had  to  present  to  the  trainee 
the  complete  range  of  problems  of  the  gunner  that  is. 
smoke,  noise,  recoil,  weight  loss,  and  the  need  to  main  - 
tain  a  sight  picture  as  the  missile  is  guided  onto  the 
target. 

The  effects  of  weight  loss  and  recoil  when  the  wea¬ 
pon  is  launched  are  simulated  by  mechanical  attach¬ 
ments  to  the  DRAGON  bipod.  Real-time  video  graphics 
are  generated  for  the  gunner's  sight  picture  and  include 
a  simulated  missile,  rocket  thruster  firings,  rocket 
exhaust  smoke,  and  launch  and  final  explosions.  Smoke 
is  simulated  by  modulating  the  background  video  level 
of  the  gunner's  sight  picture.  Sound  is  controlled  by  a 
microcomputer  interfacing  with  a  programmable  sound 
generator  to  provide  missile  gyro  wind-up.  missile 
launch  explosion,  rocket  thruster  motor  firing,  target  hit 
explosions,  and  missile  impact  explosions.  Simplified 
missile  flight  dynamics  equations  and  a  computer  pro¬ 
gram  solve  the  missile  position  along  its  flight  path, 
insert  it  into  the  gunner's  sight  picture,  and  determine 
hit  or  miss  at  the  end  of  run.  Utilization  of  actual  missile 
flight  dynamics  equations  allows  training  of  new  per¬ 
sonnel  and  refresher  training  for  experienced  gunners. 
Figure  1  is  an  artist's  concept  of  the  DRAGON  Anti- 
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i  r  conn  I  ICED  AT  GOVERNMENT  EXPENSE 


Armor  Missile  Flight  Simulator.  Army  and  Marine  Corps 
personnel  who  have  fired  the  experimental  simulator 
have  reported  it  to  be  an  excellent  trainer.  A  Marine 
Corps  Fleet  Project  Team  indicated  the  simulator  will 
increase  first  round  hits  for  the  novice  as  well  as  pro¬ 
vide  outstanding  advanced  level  training. 


Utilization 

The  trainer  incorporates  all  of  the  features  of  the 
actual  operational  missile  system,  and  can  provide  real¬ 
time  feedback  to  both  the  trainee  and  instructor  with 
regard  to  trainee  performance.  Instrumentation  is 
includgfj  in  the  simulator  launch  tube  to  monitor  how 
well  the  trainee  is  holding  down  the  launcher  prior  to 
launch.  Feedback  monitors  follow  the  progress  of  the 
missile  from  time  of  launch  to  end  of  run  and  graphi¬ 
cally  describe  the  trainee's  control  of  the  missile  flight. 
Video  monitors  also  permit  the  instructor  to  see  the 
gunner's  sight  picture  during  missile  flight.  After  a  mis¬ 
sile  flight,  a  reprise  of  the  gunner  aiming  error  or  mis¬ 
sile  flight  may  be  called  up  on  a  hardcopy  printer.  Target 
scenarios  may  also  be  varied  because  the  target  tank  is 
moved  independently  from  the  gunner’s  weapon  simu¬ 
lator;  if  the  trainee  does  not  get  set  up  and  fire  a  missile 
in  a  predetermined  time,  the  tank's  main  gun  turret  can 
be  moved  and  made  to  simulate  fire  at  the  gunner. 


Payoff 

The  DRAGON  Anti-Armor  Missile  Flight  Simulator 
will  provide  the  Army  and  Marine  Corps  with  an  effec¬ 
tive  trainer  capable  of  qualifying  novice  and  advanced 
level  gunners.  The  Army  has  noted  that  the  current 
inventory  of  similar  devices  does  not  address  the  key 
issue  of  gunner  qualification.  The  high  cost  associated 
with  firing  live  DRAGON  rounds  makes  it  financially 
impossible  to  use  that  method  to  conduct  gunner  quali¬ 
fication  and  weapon  proficiency.  The  DRAGON  trainer 
will  significantly  reduce  training  costs  by  eliminating 
the  need  for  live  missiles  and  expendable  target  vehi¬ 
cles.  In  addition,  all  trainees,  rather  then  only  a  few,  will 
get  actual  hands-on  experience  in  a  firing  scenario 
using  modified  operational  equipment  having  the  same 
weight  and  launch  sequence  characteristics  as  the  real 
weapon. 

The  DRAGON  simulator  technology  is  readily  adapt¬ 
able  to  an  entire  anti-armor  family  of  weapons.  Current 
plans  are  to  proceed  with  the  development  of  a  TOW 
heavy  anti-tank  missile  simulator.  TOW  is  similar  to 
DRAGON  but  has  three  times  the  effective  range  and  is 
usually  vehicle  mounted. 


Research  and  Development  Notes 

Development  of  this  training  system  was  a  responsi¬ 
bility  of  the  Naval  Training  Equipment  Center.  The 
responsible  researcher  was  Mr.  A.  H.  Marshall,  NTEC 
(N731),  (305)  646-5464.  Funding  was  provided  by  the 
Army  in  Program  Element  62727A,  Non-System  Train¬ 
ing  Devices  Technology,  and  by  the  Marine  Corps  in 
Program  Element  63732M,  Marine  Corps  Advanced 
Manpower  Training  Systems.  Additonal  information  ^ 
may  be  found  in  an  article  by  Marshall,  A.  H.  and  others 
entitled  “Anti-Armor  Missile  Flight  Simulator,  Pro¬ 
ceedings.  3rd  Interservice/Industry  Training  Equipment 
Conference  and  Exhibition,  American  Defense  Pre¬ 
paredness  Association,  November  30-December  2, 
1981,  pp.  288-293.  A  report  entitled  Simulated  Tank 

Anti-Armor  Gunnery  System  (STAGS-D) Dragon  Trainer, 

November  1981,  has  been  published. 

Program  dynamics  are; 


REPRODUCED  AT  GOVERNMENT  EXPENSE 


APPLICATIONS  OF  COMPUTER  VOICE  TECHNOLOGY 
Need 


The  complexity  and  sophistication  of  modern  weapon 
systems  have  significantly  increased  the  scope  of  train¬ 
ing  needed  to  provide  qualified  operators  and  maintain¬ 
ors  of  these  systems  to  the  Fleet.  The  decrease  in  oper¬ 
ational  time  devoted  to  training  caused  by  energy 
constraints  has  placed  further  demands  on  the  training 
community  not  only  to  develop  initial  skills  but  also  to 
provide  for  maintaining  proficiency  in  skills  previously 
acquired.  Further,  present  training  concepts  go  far 
beyond  individual  skill  training  to  include  team  applica¬ 
tions  involving  multi-skill  participation.  Therefore,  since 
the  mid-1 970's  there  has  been  strong  emphasis  on  the 
development  of  training  by  means  of  simulation.  Mod¬ 
ern  simulation  systems  require  greater  numbers  of  top 
quality  inductors  capable  of  assuming  more  workload 
than  has  been  previously  demanded.  Instructor  quality 
and  continuity  must  be  maintained  in  a  training  envi¬ 
ronment  faced  with  continually  increasing  demands. 

Recognizing  these  training  challenges,  considerable 
effort  has  been  focused  on  the  development  of  advanced 
training  technology.  Capabilities  exist  today  through 
automation  to  alleviate  the  increased  requirements 
being  placed  on  instructors.  Computer  voice  technology 
is  one  of  these  automated  capabilities  which  has  proved 
to  be  effective  in  several  application  areas.  The  tech¬ 
nology  has  potential  for  extensive  implementation  in 
present  and  future  training  systems. 

To  facilitate  transition  from  the  technology  base  into 
application,  a  Computer  Voice  Technology  Indoctrina¬ 
tion  Course  has  been  developed.  Its  objective  is  to  dis¬ 
seminate  information  on  the  capabilities  of  voice  tech¬ 
nology.  identify  potential  application,  and  describe 
design  procedures  for  incorporating  the  technology  in 
simulation  training  systems. 

Approach  and  Results 

Computer  voice  technology  involves  voice  recogni¬ 
tion  and  speech  generation.  Voice  recognition  technol¬ 
ogy  has  evolved  from  research  involving  computer 
recognition  of  single  letters,  to  groups  of  letters,  to  sin¬ 
gle  words,  to  groups  of  words.  The  requirement  has 
been  to  increase  the  vocabulary  capability.  Speech 
generation  capabilities  have  been  available  for  some 
time,  and  the  basic  concepts  of  this  technology  are  not 
new.  Speech  synthesizers  have  been  adapted  and  devel¬ 
oped  into  voice  response  systems  for  business,  indus¬ 
trial.  educational  and  military  applications. 

Utilizing  the  principles  of  voice  recognition  and  speech 
generation,  the  Naval  Training  Equipment  Center  devel¬ 
oped  models  which  were  incorporated  into  prototype 
training  systems,  the  Precision  Approach  Radar  and  the 
Air  Intercept  Controller  Training  Systems.  These  proto¬ 
types  were  evaluated  at  schools.  On  the  basis  of  these 
evaluations,  guidelines  were  prepared  for  incorporating 
computer  voice  technology  into  simulation  training 
systems. 

A  Landing  Signal  Officers  (LSO)  training  system  is 
now  under  consideration,  in  which  student  LSOs  will 


control  tactical  aircraft  landing  on  a  carrier.  Computer 
voice  technology  will  be  used  for  commands  and  air¬ 
craft  responses.  Figure  1  illustrates  the  concept  Addi¬ 
tionally,  computer  voice  technology  is  being  adapted  to 
augment  or  replace  conventional  instructor  functions, 
thus  easing  the  administrative  workload  and  scope  of 
instruction  required  of  the  instructor. 


The  guidelines  were  also  used  as  the  basis  of  an 
indoctrination  course  on  applications  of  voice  technol¬ 
ogy.  The  course  includes  the  following  information: 

•  Hands-on  demonstrations  illustrate  available 
technology  and  its  application. 

•  Computer  speech  generation  is  defined.  Technol¬ 
ogy  characteristics  and  the  three  types  of  speech 
generation  (digitized,  word-generated  and  phone¬ 
me-generated)  are  described.  Guidance  is  pro¬ 
vided  for  selection  of  the  type  to  be  used. 

•  Computer  voice  recognition  principles  are  pres¬ 
ented.  Emphasis  is  given  to  the  recognition  pro¬ 
cess,  entering  voice  patterns  into  the  computer, 
speaker  dependence/independence,  isolated 
word  and  connected  word  recognition,  vocabu¬ 
lary  size,  and  speech  understanding. 

•  The  principles  of  speech  generation  and  voice 
recognition  are  related  to  the  instructional  sys¬ 
tem  design  process.  Requirements  analysis,  voice 
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technology  capabilities,  operating  and  human 
factors  design  considerations,  and  preparation  of 
specifications,  are  discussed. 

Utilization 

Potential  applications  of  computer  voice  technology 
in  training  systems  include: 

•  Systems  in  which  the  voice  is  used  to  control  the 
computer  and/or  to  issue  information  from  the 
computer,  including  aircraft  cockpit  management 
and  certain  maintenance  applications. 

•  Speech  related  tasks  in  which  vocal  advisories  or 
commands  are  required,  or  there  is  voice  inter- 

■  action  such  as  in  controller  training. 

•  Providing  data  entry  in  computerized  training  sys¬ 
tems  when  hands  and  eyes  are  otherwise  occu¬ 
pied,  such  as  in  vehicle  or  aircraft  operation 
training. 

•  Systems  in  which  voice  recognition  can  be  used 
to  measure  student  performance,  such  as 
controller  training  or  command  and  control 
applications. 

O  Utilizing  speech  generation  to  ask  questions  or 
give  directions  which  require  verbal  responses 
from  the  student. 

•  Aiding  the  instructor  in  equipment  setup,  provid¬ 
ing  instruction  and  briefing  capabilities. 

O  Replacing  the  instructor  with  a  computer  model 
in  systems  requiring  high  interaction  between 
instructor  and  student. 

•  Team  training  applications  in  which  not  all  team 
members  are  present. 

•  Maintenance  training  applications  in  which  step- 
by-step  procedures  are  to  be  followed. 

Impact 

The  technology  has  high  potential  for  incorporation 
into  tactical  team  training  environments  involving 


interactive  skills.  The  computer  can  model  team  mem¬ 
bers,  substitute  for  individual  performers  and  provide 
performance  data.  Plans  are  being  developed  to  incor¬ 
porate  computer  voice  technology  into  complex  Navy 
training  systems  such  as  the  Tactical  Advanced  Combat 
Direction  and  Electronic  Warfare  (TACDEW)  System. 

Expected  results  from  continuing  research  will 
enhance  instructor  effectiveness  in  computer-based 
training  systems  by  decreasing  instructor  requirements 
and  workload;  providing  consistency  in  training;  provid¬ 
ing  freeze,  replay,  and  critique  capabilities;  and  provid¬ 
ing  automated  interaction  among  the  instructor,  the 
trainee,  and  the  training  system. 


Research  and  Development  Notes 

The  Naval  Training  Equipment  Center  has  been 
responsible  for  the  development  of  prototype  training 
systems  using  voice  technology.  Exploratory  Develop¬ 
ment  was  accomplished  under  Program  Element 
62757N,  Human  Factors  and  Simulation  Technology. 
Engineering  Development  was  sponsored  by  the  Deputy 
Chief  of  Naval  Operations  (Manpower,  Personnel  and 
Training)  (OP-01 )  in  Program  Element  64703N,  Training 
Devices  Prototype  Development,  as  projects  W0290, 
"Prototype  Air  Controller  Training  System  (PACTS)", 
and  W0291.  "Air  Intercept  Controller  Trainer  (AIC)". 
Advanced  Development  of  another  application.  Indi¬ 
vidual  Adaptive  Training  System",  W1203,  is  spon¬ 
sored  by  OP-01  in  Program  Element  63733N.  Training 
Devices  Technology.  The  responsible  researcher  is  Dr. 
R  Breaux.  NTEC  (N71).  (305)  646-5529.  The  principal 
investigators  are  Mr.  R.  Lynchard  and  Mr.  M.  Blind, 
Eagle  Technology.  Inc.,  3165  McCrory  Place,  Suite  235, 
Orlando,  FL  32803.  Project  W0290  was  previously  de¬ 
scribed  in  the  3rd  Annual  Report  of  Utilization  for 
FY1 979.  There  have  been  42  technical  reports  issued  in 
this  program.  One  of  the  more  recent  papers  describing 
the  state-of-the-art  is  "Guide  for  Voice  Technology  in 
Navy  Training  Systems,"  by  B.  Breaux,  Michael  E. 
McCauley,  and  Paul  E.  Van  Hemel,  in  the  Proceedings. 
3rd  Interservice/Industry  Conference  and  Exhibition, 
American  Defense  Preparedness  Association,  Novem¬ 
ber  30-December  2, 1981,  pp  135-142. 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 
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Program  Dynamics  are: 
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